2024-05-08T03:09:01.675 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T03:09:01.675 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_dis@teuthology --name dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} --archive /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770 -- /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/orig.config.yaml 2024-05-08T03:09:01.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T03:09:01.793 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770 branch: wip-dis-testing-squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: idryomov@gmail.com first_in_suite: false job_id: '7696770' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-dis-testing-squid ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: e0c1691aade07c0b78ccff9531f65d3c80e3e220 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: e0c1691aade07c0b78ccff9531f65d3c80e3e220 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing-squid sha1: 9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5306 sha1: e0c1691aade07c0b78ccff9531f65d3c80e3e220 sleep_before_teardown: 0 subset: 21/99 suite: rbd suite_branch: wip-dis-testing-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci suite_sha1: 9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2 targets: smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKKATMtGJ8hGd4tN4Nwj5xJG88rH0kXPsX/mOHcxW25hNQ2Sx7+crb+EQKE1pXYZsoZ+kIhMW1BRyGZ3l5XIBg= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOf+Ru6NMxzSb2AsZvdFNe9lVVa4GsCEve2nWrlJni+zZblRQg/jvlGJ8Mi9xuhplZFAxXEbqMlUjn164M4S9QY= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-07_23:02:08 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T03:09:01.793 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2/qa; will attempt to use it 2024-05-08T03:09:01.793 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2/qa/tasks 2024-05-08T03:09:01.794 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T03:09:01.795 INFO:teuthology.task.internal:Checking packages... 2024-05-08T03:09:01.820 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e0c1691aade07c0b78ccff9531f65d3c80e3e220' 2024-05-08T03:09:01.820 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T03:09:01.820 INFO:teuthology.packaging:ref: None 2024-05-08T03:09:01.820 INFO:teuthology.packaging:tag: None 2024-05-08T03:09:01.821 INFO:teuthology.packaging:branch: wip-dis-testing-squid 2024-05-08T03:09:01.821 INFO:teuthology.packaging:sha1: e0c1691aade07c0b78ccff9531f65d3c80e3e220 2024-05-08T03:09:01.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-dis-testing-squid 2024-05-08T03:09:02.043 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.ge0c1691a 2024-05-08T03:09:02.045 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T03:09:02.051 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T03:09:02.051 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T03:09:02.069 INFO:teuthology.task.internal:Saving configuration 2024-05-08T03:09:02.083 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T03:09:02.088 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T03:09:02.111 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 03:04:02.619476', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKKATMtGJ8hGd4tN4Nwj5xJG88rH0kXPsX/mOHcxW25hNQ2Sx7+crb+EQKE1pXYZsoZ+kIhMW1BRyGZ3l5XIBg='} 2024-05-08T03:09:02.133 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 03:04:02.621374', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOf+Ru6NMxzSb2AsZvdFNe9lVVa4GsCEve2nWrlJni+zZblRQg/jvlGJ8Mi9xuhplZFAxXEbqMlUjn164M4S9QY='} 2024-05-08T03:09:02.134 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T03:09:02.231 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-08T03:09:02.231 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-08T03:09:02.231 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T03:09:02.299 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fadc1bf20d0>, signals=[15]) 2024-05-08T03:09:02.299 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T03:09:02.305 INFO:teuthology.task.internal:Opening connections... 2024-05-08T03:09:02.306 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:09:02.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:09:02.384 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:09:02.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:09:02.456 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T03:09:02.463 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-05-08T03:09:02.480 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-05-08T03:09:02.481 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-05-08T03:09:02.537 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T03:09:02.538 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T03:09:02.539 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-05-08T03:09:02.559 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-05-08T03:09:02.576 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-05-08T03:09:02.577 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-05-08T03:09:02.634 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-05-08T03:09:02.634 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-05-08T03:09:02.634 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T03:09:02.635 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T03:09:02.635 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-05-08T03:09:02.667 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T03:09:02.677 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T03:09:02.684 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T03:09:02.684 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-05-08T03:09:02.687 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-05-08T03:09:02.702 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T03:09:02.708 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T03:09:02.708 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-05-08T03:09:02.744 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-05-08T03:09:03.030 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T03:09:03.091 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T03:09:03.105 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-08T03:09:03.106 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-08T03:09:03.106 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T03:09:03.107 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T03:09:03.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:09:03.123 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-05-08T03:09:03.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:09:03.124 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-05-08T03:09:03.180 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-437.el9.x86_64 2024-05-08T03:09:03.180 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-437.el9.x86_64 2024-05-08T03:09:03.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-05-08T03:09:03.181 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-437.el9.x86_64 2024-05-08T03:09:03.182 INFO:teuthology.task.kernel:Running kernel on smithi177: 5.14.0-437.el9.x86_64 2024-05-08T03:09:03.182 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2024-05-08T03:09:05.843 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.1 MB 00:01 2024-05-08T03:09:06.603 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-08T03:09:07.676 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-08T03:09:08.587 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-08T03:09:12.435 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-05-08T03:09:13.827 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-05-08T03:09:14.054 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-05-08T03:09:14.862 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-08T03:09:15.688 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-05-08T03:09:17.972 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 21 MB 00:02 2024-05-08T03:09:20.089 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-08T03:09:22.078 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T03:09:22.078 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T03:09:22.162 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:09:22.178 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:09:22.178 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-05-08T03:09:22.178 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:09:22.178 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:09:22.178 INFO:teuthology.orchestra.run.smithi177.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout:Install 4 Packages 2024-05-08T03:09:22.179 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:09:22.180 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 78 M 2024-05-08T03:09:22.180 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 123 M 2024-05-08T03:09:22.180 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:09:23.107 INFO:teuthology.orchestra.run.smithi177.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 550 kB/s | 484 kB 00:00 2024-05-08T03:09:23.139 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-05-08T03:09:23.774 INFO:teuthology.orchestra.run.smithi177.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 10 MB/s | 15 MB 00:01 2024-05-08T03:09:24.449 INFO:teuthology.orchestra.run.smithi177.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 15 MB/s | 34 MB 00:02 2024-05-08T03:09:25.103 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T03:09:25.104 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T03:09:25.191 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:09:25.204 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:09:25.204 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-05-08T03:09:25.204 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:09:25.204 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:09:25.204 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:Install 4 Packages 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 78 M 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 123 M 2024-05-08T03:09:25.205 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:09:25.374 INFO:teuthology.orchestra.run.smithi177.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-08T03:09:25.374 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:09:25.375 INFO:teuthology.orchestra.run.smithi177.stdout:Total 24 MB/s | 78 MB 00:03 2024-05-08T03:09:26.046 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:09:26.248 INFO:teuthology.orchestra.run.smithi188.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 523 kB/s | 484 kB 00:00 2024-05-08T03:09:26.391 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:09:26.392 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:09:26.798 INFO:teuthology.orchestra.run.smithi188.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 10 MB/s | 15 MB 00:01 2024-05-08T03:09:27.316 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:09:27.316 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:09:27.940 INFO:teuthology.orchestra.run.smithi188.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 13 MB/s | 34 MB 00:02 2024-05-08T03:09:28.541 INFO:teuthology.orchestra.run.smithi188.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-08T03:09:28.541 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:09:28.541 INFO:teuthology.orchestra.run.smithi188.stdout:Total 23 MB/s | 78 MB 00:03 2024-05-08T03:09:29.219 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:09:29.551 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:09:29.551 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:09:29.694 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:09:30.486 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:09:30.486 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:09:30.532 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:09:30.616 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:09:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:09:33.291 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:09:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:09:33.548 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:09:33.633 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:09:36.316 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:09:36.455 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:09:38.159 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:09:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:09:41.197 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:09:41.341 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:09:43.002 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:09:46.085 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:12.382 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:12.420 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:12.745 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:12.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:10:12.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:10:12.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:10:13.540 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:13.540 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:10:13.540 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:10:13.540 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T03:10:13.541 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T03:10:13.541 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T03:10:13.541 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T03:10:13.541 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:10:13.541 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:10:13.871 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:10:13.944 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:10:13.944 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:10:13.945 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T03:10:13.945 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T03:10:13.945 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T03:10:13.945 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T03:10:13.945 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T03:10:13.945 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:10:13.994 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:10:13.995 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:10:13.995 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.06 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 68.el9 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Group : Unspecified 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 8264254 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T03:10:14.025 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : builder@centos.org 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:10:14.026 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T03:10:14.027 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T03:10:14.027 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 5.14.0-444.el9.x86_64 2024-05-08T03:10:14.027 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T03:10:14.639 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-05-08T03:10:15.506 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:15.544 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:15.866 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:15.866 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:10:15.866 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:10:15.866 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:10:16.023 INFO:teuthology.orchestra.run.smithi177.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T03:10:16.064 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-05-08T03:10:16.067 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-05-08T03:10:16.082 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.r7rmVAlZhs 2024-05-08T03:10:16.083 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.r7rmVAlZhs 2024-05-08T03:10:16.149 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.r7rmVAlZhs 2024-05-08T03:10:16.288 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.r7rmVAlZhs is 6KB 2024-05-08T03:10:16.337 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.r7rmVAlZhs 2024-05-08T03:10:16.351 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T03:10:16.413 INFO:teuthology.orchestra.run.smithi177.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T03:10:16.413 INFO:teuthology.orchestra.run.smithi177.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T03:10:16.413 INFO:teuthology.orchestra.run.smithi177.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T03:10:16.413 INFO:teuthology.orchestra.run.smithi177.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T03:10:16.415 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T03:10:16.562 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-05-08T03:10:16.690 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:10:16.690 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:10:16.690 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:10:16.690 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T03:10:16.691 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T03:10:16.691 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T03:10:16.691 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T03:10:16.691 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:10:16.691 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:10:17.054 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:10:17.124 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:10:17.124 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:10:17.125 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T03:10:17.125 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T03:10:17.125 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T03:10:17.125 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T03:10:17.125 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T03:10:17.125 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:10:17.174 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:10:17.174 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:10:17.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2024-05-08T03:10:17.205 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-05-08T03:10:17.206 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:10:17.207 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T03:10:17.207 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T03:10:17.207 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 5.14.0-444.el9.x86_64 2024-05-08T03:10:17.208 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T03:10:17.782 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-05-08T03:10:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T03:10:19.313 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-05-08T03:10:19.315 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-05-08T03:10:19.346 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.Mrjy5XTByD 2024-05-08T03:10:19.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Mrjy5XTByD 2024-05-08T03:10:19.413 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.Mrjy5XTByD 2024-05-08T03:10:19.554 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.Mrjy5XTByD is 6KB 2024-05-08T03:10:19.574 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.Mrjy5XTByD 2024-05-08T03:10:19.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T03:10:19.650 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T03:10:19.650 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T03:10:19.650 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T03:10:19.650 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T03:10:19.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T03:10:19.785 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-05-08T03:10:46.587 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T03:10:46.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-05-08T03:10:46.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:10:49.790 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T03:10:49.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-08T03:10:49.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:11:46.647 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T03:11:49.795 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T03:11:55.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-05-08T03:11:55.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:11:56.185 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-05-08T03:11:56.668 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-05-08T03:11:56.669 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T03:11:56.669 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-05-08T03:11:56.683 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-444.el9.x86_64 2024-05-08T03:11:56.683 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T03:11:56.684 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T03:11:56.684 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T03:11:57.684 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T03:11:57.685 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T03:11:57.765 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2024-05-08T03:11:57.808 DEBUG:teuthology.parallel:result is None 2024-05-08T03:11:58.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-08T03:11:58.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:11:59.122 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-08T03:11:59.667 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-08T03:11:59.667 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-08T03:11:59.668 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-05-08T03:11:59.682 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-444.el9.x86_64 2024-05-08T03:11:59.682 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T03:11:59.682 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T03:11:59.683 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T03:12:00.684 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T03:12:00.684 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T03:12:00.765 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2024-05-08T03:12:00.810 DEBUG:teuthology.parallel:result is None 2024-05-08T03:12:00.810 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T03:12:00.817 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T03:12:00.817 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T03:12:00.820 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T03:12:00.835 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T03:12:00.841 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T03:12:00.847 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T03:12:00.847 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T03:12:00.881 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T03:12:00.912 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T03:12:00.918 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T03:12:00.919 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T03:12:00.949 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T03:12:00.983 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T03:12:00.993 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T03:12:01.075 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T03:12:01.085 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T03:12:01.087 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T03:12:01.094 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T03:12:01.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T03:12:01.096 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T03:12:01.121 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T03:12:01.130 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T03:12:01.130 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T03:12:01.165 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T03:12:01.180 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T03:12:01.263 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T03:12:01.324 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T03:12:01.408 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T03:12:01.452 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:12:01.452 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T03:12:01.520 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T03:12:01.560 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T03:12:01.607 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T03:12:01.691 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T03:12:01.730 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:12:01.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T03:12:01.796 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-05-08T03:12:01.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-05-08T03:12:01.842 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T03:12:01.925 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T03:12:02.401 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T03:12:02.409 INFO:teuthology.task.internal:Starting timer... 2024-05-08T03:12:02.410 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T03:12:02.418 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T03:12:02.427 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T03:12:02.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-05-08T03:12:02.492 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-05-08T03:12:02.494 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-05-08T03:12:02.556 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-05-08T03:12:02.558 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T03:12:02.558 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-05-08T03:12:02.582 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-05-08T03:12:02.582 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-05-08T03:12:02.604 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-05-08T03:12:02.604 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi177.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-05-08T03:12:02.605 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T03:12:02.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:12:02.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T03:12:02.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:12:02.677 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T03:12:02.677 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-05-08T03:12:02.727 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-05-08T03:12:02.757 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T03:12:02.767 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T03:12:02.772 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T03:12:02.910 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T03:12:02.925 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T03:12:02.926 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi177.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T03:16:01.760 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi177.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-05-08T03:16:01.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-05-08T03:16:01.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:16:01.840 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-05-08T03:16:01.918 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-05-08T03:16:01.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-08T03:16:01.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:16:01.993 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-08T03:16:02.078 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-08T03:16:02.079 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T03:16:02.090 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T03:16:02.090 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T03:16:02.090 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T03:16:02.093 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T03:16:02.093 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T03:16:02.120 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T03:16:02.134 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T03:16:02.148 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T03:16:02.158 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-05-08T03:16:02.165 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T03:16:02.169 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-05-08T03:16:02.182 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T03:16:02.187 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-05-08T03:16:02.196 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T03:16:02.199 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-05-08T03:16:02.213 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T03:16:02.265 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-05-08T03:16:02.266 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T03:16:02.266 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T03:16:02.266 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-05-08T03:16:02.266 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.267 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.267 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.267 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.281 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-08T03:16:02.285 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T03:16:02.285 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-08T03:16:02.285 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.285 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.285 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.285 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:16:02.286 INFO:teuthology.run_tasks:Running task install... 2024-05-08T03:16:02.294 DEBUG:teuthology.task.install:project ceph 2024-05-08T03:16:02.294 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220'}} 2024-05-08T03:16:02.294 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220'} 2024-05-08T03:16:02.294 INFO:teuthology.task.install:Using flavor: default 2024-05-08T03:16:02.300 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T03:16:02.300 INFO:teuthology.task.install:extra packages: [] 2024-05-08T03:16:02.300 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220', 'tag': None, 'wait_for_package': False} 2024-05-08T03:16:02.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0c1691aade07c0b78ccff9531f65d3c80e3e220 2024-05-08T03:16:02.303 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220', 'tag': None, 'wait_for_package': False} 2024-05-08T03:16:02.303 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0c1691aade07c0b78ccff9531f65d3c80e3e220 2024-05-08T03:16:02.455 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/ 2024-05-08T03:16:02.455 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2529.ge0c1691a 2024-05-08T03:16:02.460 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/ 2024-05-08T03:16:02.460 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2529.ge0c1691a 2024-05-08T03:16:02.581 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T03:16:02.581 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:16:02.581 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T03:16:02.587 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-dis-testing-squid/e0c1691aade07c0b78ccff9531f65d3c80e3e220/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T03:16:02.587 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:16:02.587 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T03:16:02.607 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T03:16:02.608 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e0c1691aade07c0b78ccff9531f65d3c80e3e220/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T03:16:02.613 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T03:16:02.613 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e0c1691aade07c0b78ccff9531f65d3c80e3e220/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T03:16:02.674 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T03:16:02.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T03:16:02.751 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T03:16:02.754 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T03:16:02.815 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-05-08T03:16:02.817 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-05-08T03:16:02.818 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-05-08T03:16:02.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-05-08T03:16:03.061 INFO:teuthology.orchestra.run.smithi177.stdout:54 files removed 2024-05-08T03:16:03.062 INFO:teuthology.orchestra.run.smithi188.stdout:54 files removed 2024-05-08T03:16:03.089 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw 2024-05-08T03:16:03.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw 2024-05-08T03:16:03.758 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 213 kB/s | 83 kB 00:00 2024-05-08T03:16:03.821 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 182 kB/s | 83 kB 00:00 2024-05-08T03:16:04.159 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-05-08T03:16:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-05-08T03:16:04.665 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-08T03:16:04.760 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-05-08T03:16:05.893 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.1 MB 00:01 2024-05-08T03:16:05.956 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.1 MB 00:01 2024-05-08T03:16:07.967 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-05-08T03:16:09.528 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-05-08T03:16:12.108 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-08T03:16:13.567 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-05-08T03:16:13.695 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.2 MB 00:00 2024-05-08T03:16:14.627 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-08T03:16:15.563 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-08T03:16:17.226 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-08T03:16:19.735 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-08T03:16:22.086 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 10 M 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 3.4 M 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 3.1 M 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:16:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 5.4 M 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 21 M 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 25 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 746 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 501 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 5.3 M 2024-05-08T03:16:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 45 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 135 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 161 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 321 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 301 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 99 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-05-08T03:16:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout:Install 30 Packages 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 2 Packages 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 60 M 2024-05-08T03:16:22.120 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:16:22.364 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-08T03:16:22.945 INFO:teuthology.orchestra.run.smithi188.stdout:(1/32): ceph-base-19.0.0-2529.ge0c1691a.el9.x86 8.4 MB/s | 5.4 MB 00:00 2024-05-08T03:16:22.995 INFO:teuthology.orchestra.run.smithi188.stdout:(2/32): ceph-selinux-19.0.0-2529.ge0c1691a.el9. 505 kB/s | 25 kB 00:00 2024-05-08T03:16:23.096 INFO:teuthology.orchestra.run.smithi188.stdout:(3/32): libcephfs2-19.0.0-2529.ge0c1691a.el9.x8 7.3 MB/s | 746 kB 00:00 2024-05-08T03:16:23.171 INFO:teuthology.orchestra.run.smithi188.stdout:(4/32): libradosstriper1-19.0.0-2529.ge0c1691a. 6.5 MB/s | 501 kB 00:00 2024-05-08T03:16:23.580 INFO:teuthology.orchestra.run.smithi188.stdout:(5/32): librgw2-19.0.0-2529.ge0c1691a.el9.x86_6 13 MB/s | 5.3 MB 00:00 2024-05-08T03:16:23.630 INFO:teuthology.orchestra.run.smithi188.stdout:(6/32): python3-ceph-argparse-19.0.0-2529.ge0c1 906 kB/s | 45 kB 00:00 2024-05-08T03:16:23.680 INFO:teuthology.orchestra.run.smithi188.stdout:(7/32): python3-ceph-common-19.0.0-2529.ge0c169 2.6 MB/s | 135 kB 00:00 2024-05-08T03:16:23.739 INFO:teuthology.orchestra.run.smithi188.stdout:(8/32): python3-cephfs-19.0.0-2529.ge0c1691a.el 2.7 MB/s | 161 kB 00:00 2024-05-08T03:16:23.814 INFO:teuthology.orchestra.run.smithi188.stdout:(9/32): python3-rados-19.0.0-2529.ge0c1691a.el9 4.2 MB/s | 321 kB 00:00 2024-05-08T03:16:23.889 INFO:teuthology.orchestra.run.smithi188.stdout:(10/32): python3-rbd-19.0.0-2529.ge0c1691a.el9. 3.9 MB/s | 301 kB 00:00 2024-05-08T03:16:23.956 INFO:teuthology.orchestra.run.smithi188.stdout:(11/32): python3-rgw-19.0.0-2529.ge0c1691a.el9. 1.5 MB/s | 99 kB 00:00 2024-05-08T03:16:24.182 INFO:teuthology.orchestra.run.smithi188.stdout:(12/32): ceph-common-19.0.0-2529.ge0c1691a.el9. 11 MB/s | 21 MB 00:01 2024-05-08T03:16:24.223 INFO:teuthology.orchestra.run.smithi188.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 401 kB/s | 107 kB 00:00 2024-05-08T03:16:24.266 INFO:teuthology.orchestra.run.smithi188.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-08T03:16:24.282 INFO:teuthology.orchestra.run.smithi188.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-05-08T03:16:24.374 INFO:teuthology.orchestra.run.smithi188.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-08T03:16:24.425 INFO:teuthology.orchestra.run.smithi188.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-08T03:16:24.450 INFO:teuthology.orchestra.run.smithi188.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-05-08T03:16:24.492 INFO:teuthology.orchestra.run.smithi188.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 859 kB/s | 36 kB 00:00 2024-05-08T03:16:24.526 INFO:teuthology.orchestra.run.smithi188.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-08T03:16:24.567 INFO:teuthology.orchestra.run.smithi188.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-08T03:16:24.731 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:16:24.744 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:16:24.744 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:16:24.744 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:16:24.744 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:16:24.744 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 10 M 2024-05-08T03:16:24.744 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 3.4 M 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 3.1 M 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 5.4 M 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 21 M 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 25 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 746 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T03:16:24.745 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 501 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 5.3 M 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 45 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 135 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 161 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 321 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 301 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 99 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T03:16:24.746 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout:Install 30 Packages 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 2 Packages 2024-05-08T03:16:24.747 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:24.748 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 60 M 2024-05-08T03:16:24.748 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:16:24.885 INFO:teuthology.orchestra.run.smithi188.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 861 kB/s | 309 kB 00:00 2024-05-08T03:16:24.943 INFO:teuthology.orchestra.run.smithi188.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-08T03:16:25.002 INFO:teuthology.orchestra.run.smithi188.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-05-08T03:16:25.069 INFO:teuthology.orchestra.run.smithi188.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-08T03:16:25.144 INFO:teuthology.orchestra.run.smithi188.stdout:(26/32): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-05-08T03:16:25.211 INFO:teuthology.orchestra.run.smithi188.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-08T03:16:25.320 INFO:teuthology.orchestra.run.smithi188.stdout:(28/32): ceph-radosgw-19.0.0-2529.ge0c1691a.el9 3.4 MB/s | 10 MB 00:03 2024-05-08T03:16:25.353 INFO:teuthology.orchestra.run.smithi188.stdout:(29/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 839 kB 00:00 2024-05-08T03:16:25.396 INFO:teuthology.orchestra.run.smithi188.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-05-08T03:16:25.954 INFO:teuthology.orchestra.run.smithi188.stdout:(31/32): librados2-19.0.0-2529.ge0c1691a.el9.x8 5.3 MB/s | 3.4 MB 00:00 2024-05-08T03:16:26.496 INFO:teuthology.orchestra.run.smithi188.stdout:(32/32): librbd1-19.0.0-2529.ge0c1691a.el9.x86_ 2.7 MB/s | 3.1 MB 00:01 2024-05-08T03:16:26.498 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:16:26.499 INFO:teuthology.orchestra.run.smithi188.stdout:Total 14 MB/s | 60 MB 00:04 2024-05-08T03:16:26.606 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:16:26.642 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:16:26.642 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:16:26.708 INFO:teuthology.orchestra.run.smithi177.stdout:(1/32): ceph-common-19.0.0-2529.ge0c1691a.el9.x 13 MB/s | 21 MB 00:01 2024-05-08T03:16:26.741 INFO:teuthology.orchestra.run.smithi177.stdout:(2/32): ceph-selinux-19.0.0-2529.ge0c1691a.el9. 762 kB/s | 25 kB 00:00 2024-05-08T03:16:26.842 INFO:teuthology.orchestra.run.smithi177.stdout:(3/32): libcephfs2-19.0.0-2529.ge0c1691a.el9.x8 7.3 MB/s | 746 kB 00:00 2024-05-08T03:16:26.900 INFO:teuthology.orchestra.run.smithi177.stdout:(4/32): libradosstriper1-19.0.0-2529.ge0c1691a. 8.4 MB/s | 501 kB 00:00 2024-05-08T03:16:27.217 INFO:teuthology.orchestra.run.smithi177.stdout:(5/32): librgw2-19.0.0-2529.ge0c1691a.el9.x86_6 17 MB/s | 5.3 MB 00:00 2024-05-08T03:16:27.250 INFO:teuthology.orchestra.run.smithi177.stdout:(6/32): python3-ceph-argparse-19.0.0-2529.ge0c1 1.3 MB/s | 45 kB 00:00 2024-05-08T03:16:27.270 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:16:27.271 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:16:27.301 INFO:teuthology.orchestra.run.smithi177.stdout:(7/32): python3-ceph-common-19.0.0-2529.ge0c169 2.6 MB/s | 135 kB 00:00 2024-05-08T03:16:27.370 INFO:teuthology.orchestra.run.smithi177.stdout:(8/32): ceph-base-19.0.0-2529.ge0c1691a.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-05-08T03:16:27.393 INFO:teuthology.orchestra.run.smithi177.stdout:(9/32): python3-cephfs-19.0.0-2529.ge0c1691a.el 1.7 MB/s | 161 kB 00:00 2024-05-08T03:16:27.452 INFO:teuthology.orchestra.run.smithi177.stdout:(10/32): python3-rbd-19.0.0-2529.ge0c1691a.el9. 5.0 MB/s | 301 kB 00:00 2024-05-08T03:16:27.477 INFO:teuthology.orchestra.run.smithi177.stdout:(11/32): python3-rados-19.0.0-2529.ge0c1691a.el 2.9 MB/s | 321 kB 00:00 2024-05-08T03:16:27.502 INFO:teuthology.orchestra.run.smithi177.stdout:(12/32): python3-rgw-19.0.0-2529.ge0c1691a.el9. 1.9 MB/s | 99 kB 00:00 2024-05-08T03:16:27.713 INFO:teuthology.orchestra.run.smithi177.stdout:(13/32): ceph-radosgw-19.0.0-2529.ge0c1691a.el9 3.9 MB/s | 10 MB 00:02 2024-05-08T03:16:27.778 INFO:teuthology.orchestra.run.smithi177.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 356 kB/s | 107 kB 00:00 2024-05-08T03:16:27.812 INFO:teuthology.orchestra.run.smithi177.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 518 kB/s | 160 kB 00:00 2024-05-08T03:16:27.912 INFO:teuthology.orchestra.run.smithi177.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 227 kB/s | 45 kB 00:00 2024-05-08T03:16:27.945 INFO:teuthology.orchestra.run.smithi177.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-08T03:16:27.987 INFO:teuthology.orchestra.run.smithi177.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-05-08T03:16:28.013 INFO:teuthology.orchestra.run.smithi177.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 419 kB/s | 42 kB 00:00 2024-05-08T03:16:28.038 INFO:teuthology.orchestra.run.smithi177.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-05-08T03:16:28.064 INFO:teuthology.orchestra.run.smithi177.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 474 kB/s | 36 kB 00:00 2024-05-08T03:16:28.089 INFO:teuthology.orchestra.run.smithi177.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-05-08T03:16:28.148 INFO:teuthology.orchestra.run.smithi177.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-05-08T03:16:28.181 INFO:teuthology.orchestra.run.smithi177.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.1 MB/s | 309 kB 00:00 2024-05-08T03:16:28.207 INFO:teuthology.orchestra.run.smithi177.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-05-08T03:16:28.223 INFO:teuthology.orchestra.run.smithi177.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-08T03:16:28.257 INFO:teuthology.orchestra.run.smithi177.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 5.6 MB/s | 192 kB 00:00 2024-05-08T03:16:28.316 INFO:teuthology.orchestra.run.smithi177.stdout:(28/32): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-08T03:16:28.358 INFO:teuthology.orchestra.run.smithi177.stdout:(29/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-05-08T03:16:28.400 INFO:teuthology.orchestra.run.smithi177.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-08T03:16:28.616 INFO:teuthology.orchestra.run.smithi177.stdout:(31/32): librbd1-19.0.0-2529.ge0c1691a.el9.x86_ 12 MB/s | 3.1 MB 00:00 2024-05-08T03:16:28.691 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:16:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T03:16:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T03:16:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T03:16:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T03:16:28.879 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T03:16:29.059 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T03:16:29.108 INFO:teuthology.orchestra.run.smithi177.stdout:(32/32): librados2-19.0.0-2529.ge0c1691a.el9.x8 4.3 MB/s | 3.4 MB 00:00 2024-05-08T03:16:29.110 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:16:29.111 INFO:teuthology.orchestra.run.smithi177.stdout:Total 14 MB/s | 60 MB 00:04 2024-05-08T03:16:29.112 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T03:16:29.216 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:16:29.251 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:16:29.251 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:16:29.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T03:16:29.561 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 7/34 2024-05-08T03:16:29.661 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 7/34 2024-05-08T03:16:29.749 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 8/34 2024-05-08T03:16:29.771 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 9/34 2024-05-08T03:16:29.826 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 9/34 2024-05-08T03:16:29.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T03:16:29.875 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:16:29.875 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:16:29.906 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2529.ge0c1691a.el9. 11/34 2024-05-08T03:16:29.974 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 12/34 2024-05-08T03:16:29.998 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_6 13/34 2024-05-08T03:16:30.063 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_6 13/34 2024-05-08T03:16:30.097 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T03:16:30.204 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T03:16:30.239 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T03:16:30.281 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T03:16:30.313 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T03:16:30.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T03:16:30.375 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T03:16:30.682 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T03:16:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 22/34 2024-05-08T03:16:30.854 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 22/34 2024-05-08T03:16:30.885 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 23/34 2024-05-08T03:16:30.920 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T03:16:31.282 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:16:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T03:16:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T03:16:31.390 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T03:16:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T03:16:31.416 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 26/34 2024-05-08T03:16:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T03:16:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T03:16:31.487 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 26/34 2024-05-08T03:16:31.565 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 27/34 2024-05-08T03:16:31.593 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2529.ge0c1691a.el9.x8 28/34 2024-05-08T03:16:31.679 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T03:16:31.745 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T03:16:32.129 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T03:16:32.198 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 7/34 2024-05-08T03:16:32.291 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 7/34 2024-05-08T03:16:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 8/34 2024-05-08T03:16:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 9/34 2024-05-08T03:16:32.479 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 9/34 2024-05-08T03:16:32.508 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T03:16:32.551 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.0.0-2529.ge0c1691a.el9. 11/34 2024-05-08T03:16:32.619 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 12/34 2024-05-08T03:16:32.644 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_6 13/34 2024-05-08T03:16:32.708 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_6 13/34 2024-05-08T03:16:32.742 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T03:16:32.850 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T03:16:32.885 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T03:16:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T03:16:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T03:16:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T03:16:33.020 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T03:16:33.332 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T03:16:33.375 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 22/34 2024-05-08T03:16:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 22/34 2024-05-08T03:16:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 23/34 2024-05-08T03:16:33.532 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T03:16:33.880 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 29/34 2024-05-08T03:16:33.931 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 29/34 2024-05-08T03:16:34.003 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T03:16:34.028 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 26/34 2024-05-08T03:16:34.099 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 26/34 2024-05-08T03:16:34.177 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 27/34 2024-05-08T03:16:34.205 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.0.0-2529.ge0c1691a.el9.x8 28/34 2024-05-08T03:16:34.492 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 29/34 2024-05-08T03:16:34.529 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 30/34 2024-05-08T03:16:34.617 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 30/34 2024-05-08T03:16:34.617 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T03:16:34.617 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T03:16:34.617 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:34.654 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 31/34 2024-05-08T03:16:36.442 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 29/34 2024-05-08T03:16:36.486 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 29/34 2024-05-08T03:16:37.046 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 29/34 2024-05-08T03:16:37.090 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 30/34 2024-05-08T03:16:37.188 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 30/34 2024-05-08T03:16:37.189 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T03:16:37.189 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T03:16:37.189 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:37.233 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 31/34 2024-05-08T03:16:44.383 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 31/34 2024-05-08T03:16:44.383 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-05-08T03:16:44.383 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-05-08T03:16:44.383 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-05-08T03:16:44.383 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-05-08T03:16:44.384 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-05-08T03:16:44.384 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-05-08T03:16:44.384 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-05-08T03:16:44.384 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:45.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/34 2024-05-08T03:16:45.365 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/34 2024-05-08T03:16:45.365 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:16:45.366 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T03:16:45.366 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T03:16:45.366 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T03:16:45.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:45.368 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T03:16:45.451 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T03:16:45.451 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T03:16:46.563 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 5/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_6 6/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 7/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2529.ge0c1691a.el9. 8/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2529.ge0c1691a.el9.x8 9/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 10/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 11/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 12/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 13/34 2024-05-08T03:16:46.564 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T03:16:46.565 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T03:16:46.566 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T03:16:46.566 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 31/34 2024-05-08T03:16:46.566 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T03:16:46.566 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 33/34 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 31/34 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-05-08T03:16:47.149 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-05-08T03:16:47.150 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T03:16:47.335 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T03:16:47.336 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:47.337 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:16:47.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-test 2024-05-08T03:16:48.067 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/34 2024-05-08T03:16:48.093 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/34 2024-05-08T03:16:48.093 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:16:48.093 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T03:16:48.093 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T03:16:48.093 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T03:16:48.093 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:48.095 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T03:16:48.178 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T03:16:48.178 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T03:16:48.305 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:29 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:16:48.426 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:16:48.426 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 48 M 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 164 k 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout:Install 4 Packages 2024-05-08T03:16:48.427 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:48.428 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 49 M 2024-05-08T03:16:48.428 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 207 M 2024-05-08T03:16:48.428 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:16:49.073 INFO:teuthology.orchestra.run.smithi188.stdout:(1/4): libcephsqlite-19.0.0-2529.ge0c1691a.el9. 449 kB/s | 164 kB 00:00 2024-05-08T03:16:49.386 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T03:16:49.386 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/34 2024-05-08T03:16:49.386 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/34 2024-05-08T03:16:49.386 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/34 2024-05-08T03:16:49.386 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 5/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_6 6/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 7/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2529.ge0c1691a.el9. 8/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.0.0-2529.ge0c1691a.el9.x8 9/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 10/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 11/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 12/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 13/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T03:16:49.387 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 31/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T03:16:49.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 33/34 2024-05-08T03:16:49.532 INFO:teuthology.orchestra.run.smithi188.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 370 kB/s | 305 kB 00:00 2024-05-08T03:16:49.733 INFO:teuthology.orchestra.run.smithi188.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-05-08T03:16:50.204 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T03:16:50.205 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.206 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:50.207 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:16:50.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-test 2024-05-08T03:16:51.081 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:29 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:16:51.202 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:16:51.202 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:16:51.202 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:16:51.202 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 48 M 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 164 k 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout:Install 4 Packages 2024-05-08T03:16:51.203 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:16:51.204 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 49 M 2024-05-08T03:16:51.204 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 207 M 2024-05-08T03:16:51.204 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:16:51.433 INFO:teuthology.orchestra.run.smithi177.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-05-08T03:16:51.474 INFO:teuthology.orchestra.run.smithi188.stdout:(4/4): ceph-test-19.0.0-2529.ge0c1691a.el9.x86_ 17 MB/s | 48 MB 00:02 2024-05-08T03:16:51.476 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:16:51.476 INFO:teuthology.orchestra.run.smithi188.stdout:Total 16 MB/s | 49 MB 00:03 2024-05-08T03:16:51.483 INFO:teuthology.orchestra.run.smithi177.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-08T03:16:51.504 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:16:51.509 INFO:teuthology.orchestra.run.smithi177.stdout:(3/4): libcephsqlite-19.0.0-2529.ge0c1691a.el9. 700 kB/s | 164 kB 00:00 2024-05-08T03:16:51.525 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:16:51.525 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:16:51.894 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:16:51.894 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:16:52.420 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:16:52.491 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T03:16:52.531 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T03:16:52.559 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/4 2024-05-08T03:16:55.109 INFO:teuthology.orchestra.run.smithi177.stdout:(4/4): ceph-test-19.0.0-2529.ge0c1691a.el9.x86_ 13 MB/s | 48 MB 00:03 2024-05-08T03:16:55.111 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:16:55.111 INFO:teuthology.orchestra.run.smithi177.stdout:Total 12 MB/s | 49 MB 00:03 2024-05-08T03:16:55.139 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:16:55.160 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:16:55.160 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:16:55.525 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:16:55.525 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:16:56.062 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:16:56.143 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T03:16:56.183 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T03:16:56.210 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/4 2024-05-08T03:16:57.154 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/4 2024-05-08T03:16:57.241 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/4 2024-05-08T03:16:59.586 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/4 2024-05-08T03:16:59.586 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/4 2024-05-08T03:16:59.586 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/4 2024-05-08T03:16:59.586 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T03:16:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T03:16:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:16:59.882 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:00.169 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph 2024-05-08T03:17:00.739 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:41 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:00.861 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/4 2024-05-08T03:17:00.863 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:00.866 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:00.866 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:00.866 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:00.866 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:00.866 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 6.6 k 2024-05-08T03:17:00.866 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 2.4 M 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 1.5 M 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 249 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 4.7 M 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 17 M 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T03:17:00.867 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T03:17:00.868 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout:Install 36 Packages 2024-05-08T03:17:00.869 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:00.870 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 30 M 2024-05-08T03:17:00.870 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 107 M 2024-05-08T03:17:00.870 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:00.942 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/4 2024-05-08T03:17:01.594 INFO:teuthology.orchestra.run.smithi188.stdout:(1/36): ceph-19.0.0-2529.ge0c1691a.el9.x86_64.r 25 kB/s | 6.6 kB 00:00 2024-05-08T03:17:01.728 INFO:teuthology.orchestra.run.smithi188.stdout:(2/36): ceph-mds-19.0.0-2529.ge0c1691a.el9.x86_ 6.0 MB/s | 2.4 MB 00:00 2024-05-08T03:17:01.944 INFO:teuthology.orchestra.run.smithi188.stdout:(3/36): ceph-mgr-19.0.0-2529.ge0c1691a.el9.x86_ 2.4 MB/s | 1.5 MB 00:00 2024-05-08T03:17:02.028 INFO:teuthology.orchestra.run.smithi188.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2529.ge0c1 2.9 MB/s | 249 kB 00:00 2024-05-08T03:17:02.178 INFO:teuthology.orchestra.run.smithi188.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-05-08T03:17:02.295 INFO:teuthology.orchestra.run.smithi188.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 11 MB/s | 1.2 MB 00:00 2024-05-08T03:17:02.345 INFO:teuthology.orchestra.run.smithi188.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T03:17:02.395 INFO:teuthology.orchestra.run.smithi188.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-05-08T03:17:02.579 INFO:teuthology.orchestra.run.smithi188.stdout:(9/36): ceph-osd-19.0.0-2529.ge0c1691a.el9.x86_ 20 MB/s | 17 MB 00:00 2024-05-08T03:17:02.604 INFO:teuthology.orchestra.run.smithi188.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 599 kB/s | 125 kB 00:00 2024-05-08T03:17:02.738 INFO:teuthology.orchestra.run.smithi188.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-05-08T03:17:03.272 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/4 2024-05-08T03:17:03.273 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/4 2024-05-08T03:17:03.273 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/4 2024-05-08T03:17:03.273 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T03:17:03.280 INFO:teuthology.orchestra.run.smithi188.stdout:(12/36): ceph-mon-19.0.0-2529.ge0c1691a.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-05-08T03:17:03.447 INFO:teuthology.orchestra.run.smithi188.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 224 kB/s | 188 kB 00:00 2024-05-08T03:17:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T03:17:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:03.549 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T03:17:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T03:17:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:03.551 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:03.580 INFO:teuthology.orchestra.run.smithi188.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 204 kB/s | 172 kB 00:00 2024-05-08T03:17:03.859 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph 2024-05-08T03:17:03.931 INFO:teuthology.orchestra.run.smithi188.stdout:(15/36): python3-markupsafe-1.1.1-12.el9.x86_64 53 kB/s | 35 kB 00:00 2024-05-08T03:17:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 72 kB/s | 51 kB 00:00 2024-05-08T03:17:04.315 INFO:teuthology.orchestra.run.smithi188.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 57 kB/s | 42 kB 00:00 2024-05-08T03:17:04.434 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:42 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:04.540 INFO:teuthology.orchestra.run.smithi188.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 394 kB/s | 151 kB 00:00 2024-05-08T03:17:04.558 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:04.560 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:04.560 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 6.6 k 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 2.4 M 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 1.5 M 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 249 k 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 4.7 M 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 17 M 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T03:17:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T03:17:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T03:17:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout:Install 36 Packages 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 30 M 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 107 M 2024-05-08T03:17:04.564 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:04.582 INFO:teuthology.orchestra.run.smithi188.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 163 kB/s | 43 kB 00:00 2024-05-08T03:17:04.607 INFO:teuthology.orchestra.run.smithi188.stdout:(20/36): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-05-08T03:17:04.699 INFO:teuthology.orchestra.run.smithi188.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-08T03:17:04.791 INFO:teuthology.orchestra.run.smithi188.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 193 kB/s | 18 kB 00:00 2024-05-08T03:17:04.833 INFO:teuthology.orchestra.run.smithi188.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 47 kB/s | 11 kB 00:00 2024-05-08T03:17:04.867 INFO:teuthology.orchestra.run.smithi188.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-05-08T03:17:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.1 MB/s | 358 kB 00:00 2024-05-08T03:17:04.917 INFO:teuthology.orchestra.run.smithi188.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-05-08T03:17:04.943 INFO:teuthology.orchestra.run.smithi188.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-05-08T03:17:04.984 INFO:teuthology.orchestra.run.smithi188.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 503 kB/s | 46 kB 00:00 2024-05-08T03:17:05.043 INFO:teuthology.orchestra.run.smithi188.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 628 kB/s | 79 kB 00:00 2024-05-08T03:17:05.068 INFO:teuthology.orchestra.run.smithi188.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 196 kB/s | 16 kB 00:00 2024-05-08T03:17:05.093 INFO:teuthology.orchestra.run.smithi188.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-08T03:17:05.135 INFO:teuthology.orchestra.run.smithi188.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2024-05-08T03:17:05.161 INFO:teuthology.orchestra.run.smithi188.stdout:(33/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 765 kB/s | 90 kB 00:00 2024-05-08T03:17:05.219 INFO:teuthology.orchestra.run.smithi188.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 343 kB/s | 20 kB 00:00 2024-05-08T03:17:05.245 INFO:teuthology.orchestra.run.smithi188.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-05-08T03:17:05.319 INFO:teuthology.orchestra.run.smithi177.stdout:(1/36): ceph-19.0.0-2529.ge0c1691a.el9.x86_64.r 40 kB/s | 6.6 kB 00:00 2024-05-08T03:17:05.353 INFO:teuthology.orchestra.run.smithi188.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2024-05-08T03:17:05.359 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:05.360 INFO:teuthology.orchestra.run.smithi188.stdout:Total 6.7 MB/s | 30 MB 00:04 2024-05-08T03:17:05.505 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:05.546 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:05.546 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:05.595 INFO:teuthology.orchestra.run.smithi177.stdout:(2/36): ceph-mds-19.0.0-2529.ge0c1691a.el9.x86_ 5.4 MB/s | 2.4 MB 00:00 2024-05-08T03:17:05.770 INFO:teuthology.orchestra.run.smithi177.stdout:(3/36): ceph-mon-19.0.0-2529.ge0c1691a.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-05-08T03:17:05.854 INFO:teuthology.orchestra.run.smithi177.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2529.ge0c1 2.9 MB/s | 249 kB 00:00 2024-05-08T03:17:05.962 INFO:teuthology.orchestra.run.smithi177.stdout:(5/36): ceph-mgr-19.0.0-2529.ge0c1691a.el9.x86_ 1.9 MB/s | 1.5 MB 00:00 2024-05-08T03:17:05.963 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:05.963 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:06.146 INFO:teuthology.orchestra.run.smithi177.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 868 kB/s | 253 kB 00:00 2024-05-08T03:17:06.246 INFO:teuthology.orchestra.run.smithi177.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-08T03:17:06.438 INFO:teuthology.orchestra.run.smithi177.stdout:(8/36): ceph-osd-19.0.0-2529.ge0c1691a.el9.x86_ 20 MB/s | 17 MB 00:00 2024-05-08T03:17:06.463 INFO:teuthology.orchestra.run.smithi177.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 624 kB/s | 135 kB 00:00 2024-05-08T03:17:06.622 INFO:teuthology.orchestra.run.smithi177.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-05-08T03:17:06.672 INFO:teuthology.orchestra.run.smithi177.stdout:(11/36): python3-requests-2.25.1-8.el9.noarch.r 534 kB/s | 125 kB 00:00 2024-05-08T03:17:06.714 INFO:teuthology.orchestra.run.smithi177.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 1.6 MB/s | 1.2 MB 00:00 2024-05-08T03:17:06.739 INFO:teuthology.orchestra.run.smithi177.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-08T03:17:06.765 INFO:teuthology.orchestra.run.smithi177.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-08T03:17:06.790 INFO:teuthology.orchestra.run.smithi177.stdout:(15/36): python3-markupsafe-1.1.1-12.el9.x86_64 460 kB/s | 35 kB 00:00 2024-05-08T03:17:06.815 INFO:teuthology.orchestra.run.smithi177.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-05-08T03:17:06.840 INFO:teuthology.orchestra.run.smithi177.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-05-08T03:17:06.882 INFO:teuthology.orchestra.run.smithi177.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-08T03:17:07.016 INFO:teuthology.orchestra.run.smithi177.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 248 kB/s | 43 kB 00:00 2024-05-08T03:17:07.066 INFO:teuthology.orchestra.run.smithi177.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 603 kB/s | 151 kB 00:00 2024-05-08T03:17:07.108 INFO:teuthology.orchestra.run.smithi177.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 762 kB/s | 172 kB 00:00 2024-05-08T03:17:07.133 INFO:teuthology.orchestra.run.smithi177.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-05-08T03:17:07.167 INFO:teuthology.orchestra.run.smithi177.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-05-08T03:17:07.192 INFO:teuthology.orchestra.run.smithi177.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 396 kB/s | 23 kB 00:00 2024-05-08T03:17:07.217 INFO:teuthology.orchestra.run.smithi177.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-05-08T03:17:07.243 INFO:teuthology.orchestra.run.smithi177.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-05-08T03:17:07.245 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:07.268 INFO:teuthology.orchestra.run.smithi177.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-05-08T03:17:07.293 INFO:teuthology.orchestra.run.smithi177.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 612 kB/s | 46 kB 00:00 2024-05-08T03:17:07.314 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T03:17:07.319 INFO:teuthology.orchestra.run.smithi177.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-05-08T03:17:07.344 INFO:teuthology.orchestra.run.smithi177.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 325 kB/s | 16 kB 00:00 2024-05-08T03:17:07.346 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T03:17:07.374 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T03:17:07.378 INFO:teuthology.orchestra.run.smithi177.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-08T03:17:07.403 INFO:teuthology.orchestra.run.smithi177.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 610 kB/s | 36 kB 00:00 2024-05-08T03:17:07.420 INFO:teuthology.orchestra.run.smithi177.stdout:(33/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 892 kB/s | 90 kB 00:00 2024-05-08T03:17:07.445 INFO:teuthology.orchestra.run.smithi177.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-05-08T03:17:07.447 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T03:17:07.479 INFO:teuthology.orchestra.run.smithi177.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-05-08T03:17:07.499 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T03:17:07.521 INFO:teuthology.orchestra.run.smithi177.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-05-08T03:17:07.522 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:07.522 INFO:teuthology.orchestra.run.smithi177.stdout:Total 10 MB/s | 30 MB 00:02 2024-05-08T03:17:07.532 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T03:17:07.625 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T03:17:07.667 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:07.706 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:07.706 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:07.710 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T03:17:07.751 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T03:17:07.786 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T03:17:07.816 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T03:17:07.850 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T03:17:07.884 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T03:17:07.924 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T03:17:07.967 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T03:17:08.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T03:17:08.049 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T03:17:08.123 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T03:17:08.146 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:08.146 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:08.261 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T03:17:08.349 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T03:17:08.401 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T03:17:08.449 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T03:17:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T03:17:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T03:17:08.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T03:17:08.812 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T03:17:08.891 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T03:17:08.987 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T03:17:09.028 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T03:17:09.146 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T03:17:09.300 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2529.ge0c1691a.el9. 31/36 2024-05-08T03:17:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/36 2024-05-08T03:17:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/36 2024-05-08T03:17:09.356 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:09.356 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T03:17:09.356 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T03:17:09.356 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T03:17:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:09.465 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T03:17:09.497 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T03:17:09.526 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T03:17:09.597 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T03:17:09.641 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T03:17:09.682 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T03:17:09.774 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T03:17:09.861 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T03:17:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T03:17:09.945 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T03:17:09.975 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T03:17:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T03:17:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T03:17:10.084 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T03:17:10.118 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T03:17:10.153 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T03:17:10.183 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T03:17:10.264 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T03:17:10.410 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T03:17:10.491 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T03:17:10.544 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T03:17:10.583 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T03:17:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T03:17:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T03:17:10.797 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 33/36 2024-05-08T03:17:10.825 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 33/36 2024-05-08T03:17:10.825 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:10.825 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T03:17:10.825 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T03:17:10.825 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T03:17:10.825 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:10.880 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T03:17:10.929 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T03:17:11.007 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T03:17:11.095 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T03:17:11.137 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T03:17:11.251 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T03:17:11.273 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 34/36 2024-05-08T03:17:11.297 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 34/36 2024-05-08T03:17:11.297 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:11.297 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T03:17:11.297 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T03:17:11.298 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T03:17:11.298 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:11.409 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2529.ge0c1691a.el9. 31/36 2024-05-08T03:17:11.439 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/36 2024-05-08T03:17:11.464 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 32/36 2024-05-08T03:17:11.464 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:11.464 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T03:17:11.464 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T03:17:11.464 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T03:17:11.464 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:11.523 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 35/36 2024-05-08T03:17:11.547 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 35/36 2024-05-08T03:17:11.548 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:11.548 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T03:17:11.548 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T03:17:11.548 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T03:17:11.548 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:11.612 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 36/36 2024-05-08T03:17:12.968 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 33/36 2024-05-08T03:17:12.998 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 33/36 2024-05-08T03:17:12.998 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:12.998 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T03:17:12.998 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T03:17:12.998 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T03:17:12.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:13.444 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 34/36 2024-05-08T03:17:13.455 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 36/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 5/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2529.ge0c1691a.el9. 6/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T03:17:13.456 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T03:17:13.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T03:17:13.458 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T03:17:13.458 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T03:17:13.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T03:17:13.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T03:17:13.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T03:17:13.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T03:17:13.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T03:17:13.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T03:17:13.467 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 34/36 2024-05-08T03:17:13.467 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:13.468 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T03:17:13.468 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T03:17:13.468 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T03:17:13.468 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 35/36 2024-05-08T03:17:13.719 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 35/36 2024-05-08T03:17:13.719 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:13.719 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T03:17:13.719 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T03:17:13.719 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T03:17:13.719 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:13.783 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 36/36 2024-05-08T03:17:14.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T03:17:14.475 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T03:17:14.476 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:14.477 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:14.858 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-base 2024-05-08T03:17:15.476 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:56 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 36/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 3/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 4/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 5/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2529.ge0c1691a.el9. 6/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T03:17:15.543 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T03:17:15.544 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T03:17:15.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T03:17:15.560 INFO:teuthology.orchestra.run.smithi188.stdout:Package ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:17:15.605 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:15.606 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:17:15.606 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:15.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install cephadm 2024-05-08T03:17:16.256 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:57 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:16.378 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 763 k 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 763 k 2024-05-08T03:17:16.379 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 769 k 2024-05-08T03:17:16.380 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:16.545 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T03:17:16.545 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T03:17:16.546 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T03:17:16.547 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:16.548 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:16.901 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-base 2024-05-08T03:17:17.149 INFO:teuthology.orchestra.run.smithi188.stdout:cephadm-19.0.0-2529.ge0c1691a.el9.noarch.rpm 991 kB/s | 763 kB 00:00 2024-05-08T03:17:17.150 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:17.150 INFO:teuthology.orchestra.run.smithi188.stdout:Total 989 kB/s | 763 kB 00:00 2024-05-08T03:17:17.151 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:17.156 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:17.156 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:17.162 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:17.163 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:17.239 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:17.479 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:55 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:17.559 INFO:teuthology.orchestra.run.smithi177.stdout:Package ceph-base-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:17:17.602 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:17.603 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:17:17.603 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:17.653 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:17.670 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install cephadm 2024-05-08T03:17:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:18.114 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:18.236 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:56 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:18.359 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:18.362 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:18.362 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:18.362 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:18.362 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:18.362 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 763 k 2024-05-08T03:17:18.362 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 763 k 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 769 k 2024-05-08T03:17:18.363 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:18.393 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:18.393 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:18.393 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:18.393 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:18.393 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:18.394 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:18.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T03:17:19.189 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:00 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:19.315 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repo Size 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 145 k 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:19.316 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:17:19.317 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:19.317 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 145 k 2024-05-08T03:17:19.317 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 435 k 2024-05-08T03:17:19.317 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:19.378 INFO:teuthology.orchestra.run.smithi177.stdout:cephadm-19.0.0-2529.ge0c1691a.el9.noarch.rpm 749 kB/s | 763 kB 00:01 2024-05-08T03:17:19.380 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:19.380 INFO:teuthology.orchestra.run.smithi177.stdout:Total 748 kB/s | 763 kB 00:01 2024-05-08T03:17:19.380 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:19.385 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:19.385 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:19.391 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:19.391 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:19.468 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:19.944 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:20.023 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-immutable-object-cache-19.0.0-2529.ge0c169 205 kB/s | 145 kB 00:00 2024-05-08T03:17:20.023 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:20.023 INFO:teuthology.orchestra.run.smithi188.stdout:Total 205 kB/s | 145 kB 00:00 2024-05-08T03:17:20.024 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:20.024 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:20.031 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:20.031 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:20.089 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:20.089 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:20.242 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:20.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a. 1/1 2024-05-08T03:17:20.366 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a. 1/1 2024-05-08T03:17:20.366 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:20.366 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T03:17:20.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:20.379 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:20.624 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/1 2024-05-08T03:17:20.624 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:20.624 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:20.624 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:20.624 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:20.624 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:20.829 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T03:17:21.250 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a. 1/1 2024-05-08T03:17:21.250 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:21.250 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:21.250 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:21.250 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:21.250 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:21.400 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:59 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:21.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-mgr 2024-05-08T03:17:21.523 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:21.524 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-05-08T03:17:21.524 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:21.524 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 145 k 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 145 k 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 435 k 2024-05-08T03:17:21.525 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:22.054 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:03 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:22.054 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-immutable-object-cache-19.0.0-2529.ge0c169 275 kB/s | 145 kB 00:00 2024-05-08T03:17:22.054 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:22.054 INFO:teuthology.orchestra.run.smithi177.stdout:Total 274 kB/s | 145 kB 00:00 2024-05-08T03:17:22.054 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:22.061 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:22.061 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:22.118 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:22.118 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:22.134 INFO:teuthology.orchestra.run.smithi188.stdout:Package ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:17:22.179 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:22.180 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:17:22.180 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:22.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T03:17:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a. 1/1 2024-05-08T03:17:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a. 1/1 2024-05-08T03:17:22.379 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:22.379 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T03:17:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:22.801 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:03 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:22.924 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:22.925 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:22.925 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:22.925 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:22.925 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 3.5 M 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 32 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 16 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T03:17:22.926 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout:Install 12 Packages 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:22.927 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 8.6 M 2024-05-08T03:17:22.928 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 103 M 2024-05-08T03:17:22.928 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:23.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a. 1/1 2024-05-08T03:17:23.247 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:23.248 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:23.248 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:23.248 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:23.248 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:23.475 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr 2024-05-08T03:17:23.695 INFO:teuthology.orchestra.run.smithi188.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2529.ge0c 44 kB/s | 16 kB 00:00 2024-05-08T03:17:23.721 INFO:teuthology.orchestra.run.smithi188.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2529.ge0 80 kB/s | 32 kB 00:00 2024-05-08T03:17:24.043 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:24.125 INFO:teuthology.orchestra.run.smithi177.stdout:Package ceph-mgr-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:17:24.169 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:24.170 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:17:24.171 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:24.234 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T03:17:24.587 INFO:teuthology.orchestra.run.smithi188.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 311 kB/s | 269 kB 00:00 2024-05-08T03:17:24.763 INFO:teuthology.orchestra.run.smithi188.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 968 kB/s | 1.0 MB 00:01 2024-05-08T03:17:24.800 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:24.922 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:24.923 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:24.923 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:24.923 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 3.5 M 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 32 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 16 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T03:17:24.924 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout:Install 12 Packages 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 8.6 M 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 103 M 2024-05-08T03:17:24.925 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:24.996 INFO:teuthology.orchestra.run.smithi188.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.3 MB/s | 548 kB 00:00 2024-05-08T03:17:25.055 INFO:teuthology.orchestra.run.smithi188.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-05-08T03:17:25.180 INFO:teuthology.orchestra.run.smithi188.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 16 MB/s | 2.0 MB 00:00 2024-05-08T03:17:25.230 INFO:teuthology.orchestra.run.smithi188.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-05-08T03:17:25.289 INFO:teuthology.orchestra.run.smithi188.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 526 kB/s | 31 kB 00:00 2024-05-08T03:17:25.348 INFO:teuthology.orchestra.run.smithi188.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-05-08T03:17:25.406 INFO:teuthology.orchestra.run.smithi188.stdout:(11/12): ceph-mgr-dashboard-19.0.0-2529.ge0c169 1.7 MB/s | 3.5 MB 00:02 2024-05-08T03:17:25.473 INFO:teuthology.orchestra.run.smithi188.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 975 kB/s | 863 kB 00:00 2024-05-08T03:17:25.475 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:25.476 INFO:teuthology.orchestra.run.smithi188.stdout:Total 3.4 MB/s | 8.6 MB 00:02 2024-05-08T03:17:25.525 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:25.543 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:25.543 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:25.721 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:25.722 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:25.906 INFO:teuthology.orchestra.run.smithi177.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2529.ge0c 40 kB/s | 16 kB 00:00 2024-05-08T03:17:25.998 INFO:teuthology.orchestra.run.smithi177.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2529.ge0 63 kB/s | 32 kB 00:00 2024-05-08T03:17:26.360 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:26.425 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T03:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T03:17:26.506 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T03:17:26.545 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T03:17:26.647 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T03:17:26.771 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T03:17:26.848 INFO:teuthology.orchestra.run.smithi177.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 317 kB/s | 269 kB 00:00 2024-05-08T03:17:26.864 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T03:17:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T03:17:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T03:17:26.965 INFO:teuthology.orchestra.run.smithi177.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 975 kB/s | 1.0 MB 00:01 2024-05-08T03:17:26.976 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2529.ge0c1691a.el9 10/12 2024-05-08T03:17:27.099 INFO:teuthology.orchestra.run.smithi177.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.4 MB/s | 863 kB 00:00 2024-05-08T03:17:27.166 INFO:teuthology.orchestra.run.smithi177.stdout:(6/12): ceph-mgr-dashboard-19.0.0-2529.ge0c1691 2.1 MB/s | 3.5 MB 00:01 2024-05-08T03:17:27.233 INFO:teuthology.orchestra.run.smithi177.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 145 kB/s | 19 kB 00:00 2024-05-08T03:17:27.316 INFO:teuthology.orchestra.run.smithi177.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.5 MB/s | 548 kB 00:00 2024-05-08T03:17:27.341 INFO:teuthology.orchestra.run.smithi177.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.3 MB/s | 144 kB 00:00 2024-05-08T03:17:27.383 INFO:teuthology.orchestra.run.smithi177.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 460 kB/s | 31 kB 00:00 2024-05-08T03:17:27.408 INFO:teuthology.orchestra.run.smithi177.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-05-08T03:17:27.542 INFO:teuthology.orchestra.run.smithi177.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.4 MB/s | 2.0 MB 00:00 2024-05-08T03:17:27.544 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:27.544 INFO:teuthology.orchestra.run.smithi177.stdout:Total 3.3 MB/s | 8.6 MB 00:02 2024-05-08T03:17:27.595 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:27.613 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:27.613 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:27.789 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:27.790 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:28.013 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2529.ge0c1691a.el 11/12 2024-05-08T03:17:28.043 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noa 12/12 2024-05-08T03:17:28.439 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:28.510 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T03:17:28.553 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T03:17:28.600 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T03:17:28.630 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T03:17:28.733 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T03:17:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T03:17:28.975 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T03:17:29.025 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T03:17:29.064 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T03:17:29.096 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2529.ge0c1691a.el9 10/12 2024-05-08T03:17:29.324 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noa 12/12 2024-05-08T03:17:29.324 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2529.ge0c1691a.el 1/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noa 2/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2529.ge0c1691a.el9 3/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T03:17:29.325 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T03:17:29.625 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T03:17:29.625 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:29.625 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:29.626 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:29.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T03:17:30.138 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2529.ge0c1691a.el 11/12 2024-05-08T03:17:30.179 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noa 12/12 2024-05-08T03:17:30.420 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:11 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:30.541 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 7.4 M 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:30.542 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout:Install 12 Packages 2024-05-08T03:17:30.543 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:30.544 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 38 M 2024-05-08T03:17:30.544 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 209 M 2024-05-08T03:17:30.544 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:31.538 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noa 12/12 2024-05-08T03:17:31.538 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2529.ge0c1691a.el 1/12 2024-05-08T03:17:31.538 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noa 2/12 2024-05-08T03:17:31.538 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2529.ge0c1691a.el9 3/12 2024-05-08T03:17:31.538 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T03:17:31.539 INFO:teuthology.orchestra.run.smithi188.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 244 kB/s | 195 kB 00:00 2024-05-08T03:17:31.639 INFO:teuthology.orchestra.run.smithi188.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 33 kB 00:00 2024-05-08T03:17:31.690 INFO:teuthology.orchestra.run.smithi188.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 843 kB/s | 803 kB 00:00 2024-05-08T03:17:31.765 INFO:teuthology.orchestra.run.smithi188.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 244 kB/s | 18 kB 00:00 2024-05-08T03:17:31.840 INFO:teuthology.orchestra.run.smithi188.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:31.856 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:31.865 INFO:teuthology.orchestra.run.smithi188.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-08T03:17:31.949 INFO:teuthology.orchestra.run.smithi188.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-08T03:17:32.024 INFO:teuthology.orchestra.run.smithi188.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-05-08T03:17:32.075 INFO:teuthology.orchestra.run.smithi188.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-05-08T03:17:32.109 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T03:17:32.150 INFO:teuthology.orchestra.run.smithi188.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-05-08T03:17:32.617 INFO:teuthology.orchestra.run.smithi188.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-05-08T03:17:32.671 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:10 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:32.717 INFO:teuthology.orchestra.run.smithi188.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 3.7 MB/s | 7.4 MB 00:01 2024-05-08T03:17:32.718 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:32.718 INFO:teuthology.orchestra.run.smithi188.stdout:Total 17 MB/s | 38 MB 00:02 2024-05-08T03:17:32.793 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:32.794 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-05-08T03:17:32.794 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:32.794 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-05-08T03:17:32.794 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 7.4 M 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T03:17:32.795 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout:Install 12 Packages 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 38 M 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 209 M 2024-05-08T03:17:32.796 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:33.075 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:33.101 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:33.101 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:33.426 INFO:teuthology.orchestra.run.smithi177.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 864 kB/s | 195 kB 00:00 2024-05-08T03:17:33.518 INFO:teuthology.orchestra.run.smithi177.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-05-08T03:17:33.530 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:33.531 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:33.651 INFO:teuthology.orchestra.run.smithi177.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-05-08T03:17:33.735 INFO:teuthology.orchestra.run.smithi177.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-05-08T03:17:33.818 INFO:teuthology.orchestra.run.smithi177.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-05-08T03:17:33.911 INFO:teuthology.orchestra.run.smithi177.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.6 MB/s | 3.0 MB 00:00 2024-05-08T03:17:33.994 INFO:teuthology.orchestra.run.smithi177.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-08T03:17:34.244 INFO:teuthology.orchestra.run.smithi177.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-25 7.1 MB/s | 7.4 MB 00:01 2024-05-08T03:17:34.293 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:34.311 INFO:teuthology.orchestra.run.smithi177.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-08T03:17:34.363 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T03:17:34.395 INFO:teuthology.orchestra.run.smithi177.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.6 MB/s | 250 kB 00:00 2024-05-08T03:17:34.406 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T03:17:34.506 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T03:17:34.541 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T03:17:34.562 INFO:teuthology.orchestra.run.smithi177.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 6.4 MB/s | 4.7 MB 00:00 2024-05-08T03:17:34.718 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T03:17:34.748 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T03:17:34.824 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T03:17:35.229 INFO:teuthology.orchestra.run.smithi177.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-05-08T03:17:35.229 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:35.230 INFO:teuthology.orchestra.run.smithi177.stdout:Total 16 MB/s | 38 MB 00:02 2024-05-08T03:17:35.467 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T03:17:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T03:17:35.585 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:35.612 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:35.612 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:36.045 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:36.045 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T03:17:36.799 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:36.854 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T03:17:36.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T03:17:36.992 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T03:17:37.028 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T03:17:37.205 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T03:17:37.244 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T03:17:37.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T03:17:37.836 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T03:17:37.872 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c16 12/12 2024-05-08T03:17:37.965 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T03:17:38.081 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T03:17:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T03:17:40.320 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c16 12/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c16 1/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T03:17:40.321 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T03:17:40.357 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T03:17:40.387 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c16 12/12 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T03:17:40.745 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T03:17:40.746 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T03:17:40.746 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T03:17:40.746 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T03:17:40.746 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:40.746 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:41.089 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-mgr-rook 2024-05-08T03:17:41.665 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:22 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:41.788 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 49 k 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T03:17:41.789 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout:Install 13 Packages 2024-05-08T03:17:41.790 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:41.791 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 2.2 M 2024-05-08T03:17:41.791 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 26 M 2024-05-08T03:17:41.791 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:42.155 INFO:teuthology.orchestra.run.smithi188.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 206 kB/s | 19 kB 00:00 2024-05-08T03:17:42.180 INFO:teuthology.orchestra.run.smithi188.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 217 kB/s | 26 kB 00:00 2024-05-08T03:17:42.264 INFO:teuthology.orchestra.run.smithi188.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-05-08T03:17:42.297 INFO:teuthology.orchestra.run.smithi188.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-08T03:17:42.331 INFO:teuthology.orchestra.run.smithi188.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 279 kB 00:00 2024-05-08T03:17:42.356 INFO:teuthology.orchestra.run.smithi188.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-05-08T03:17:42.381 INFO:teuthology.orchestra.run.smithi188.stdout:(7/13): ceph-mgr-rook-19.0.0-2529.ge0c1691a.el9 154 kB/s | 49 kB 00:00 2024-05-08T03:17:42.465 INFO:teuthology.orchestra.run.smithi188.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 138 kB/s | 15 kB 00:00 2024-05-08T03:17:42.482 INFO:teuthology.orchestra.run.smithi188.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 213 kB/s | 32 kB 00:00 2024-05-08T03:17:42.583 INFO:teuthology.orchestra.run.smithi188.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-05-08T03:17:42.607 INFO:teuthology.orchestra.run.smithi188.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 910 kB/s | 206 kB 00:00 2024-05-08T03:17:42.666 INFO:teuthology.orchestra.run.smithi188.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-05-08T03:17:42.724 INFO:teuthology.orchestra.run.smithi188.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.9 MB/s | 1.0 MB 00:00 2024-05-08T03:17:42.725 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:42.725 INFO:teuthology.orchestra.run.smithi188.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-05-08T03:17:42.790 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:42.804 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:42.804 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:42.888 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:42.889 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c16 12/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c16 1/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T03:17:42.940 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T03:17:43.388 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:43.389 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:43.396 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:43.554 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T03:17:43.608 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T03:17:43.664 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T03:17:43.701 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T03:17:43.742 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T03:17:43.751 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-rook 2024-05-08T03:17:43.834 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T03:17:43.937 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T03:17:43.986 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T03:17:44.333 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:22 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:44.462 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:44.463 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:44.463 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:44.463 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:44.463 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:44.463 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 49 k 2024-05-08T03:17:44.463 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:44.464 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:44.465 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:44.465 INFO:teuthology.orchestra.run.smithi177.stdout:Install 13 Packages 2024-05-08T03:17:44.465 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:44.465 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 2.2 M 2024-05-08T03:17:44.465 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 26 M 2024-05-08T03:17:44.465 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:44.694 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T03:17:44.741 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T03:17:44.775 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T03:17:44.823 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T03:17:44.853 INFO:teuthology.orchestra.run.smithi177.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 279 kB/s | 26 kB 00:00 2024-05-08T03:17:44.853 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 13/13 2024-05-08T03:17:44.878 INFO:teuthology.orchestra.run.smithi177.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 164 kB/s | 19 kB 00:00 2024-05-08T03:17:45.012 INFO:teuthology.orchestra.run.smithi177.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-05-08T03:17:45.037 INFO:teuthology.orchestra.run.smithi177.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-08T03:17:45.087 INFO:teuthology.orchestra.run.smithi177.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-05-08T03:17:45.113 INFO:teuthology.orchestra.run.smithi177.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-05-08T03:17:45.305 INFO:teuthology.orchestra.run.smithi177.stdout:(7/13): ceph-mgr-rook-19.0.0-2529.ge0c1691a.el9 91 kB/s | 49 kB 00:00 2024-05-08T03:17:45.580 INFO:teuthology.orchestra.run.smithi177.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 65 kB/s | 32 kB 00:00 2024-05-08T03:17:45.599 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 13/13 2024-05-08T03:17:45.599 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 1/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T03:17:45.600 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T03:17:45.606 INFO:teuthology.orchestra.run.smithi177.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 30 kB/s | 15 kB 00:00 2024-05-08T03:17:45.889 INFO:teuthology.orchestra.run.smithi177.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 209 kB/s | 59 kB 00:00 2024-05-08T03:17:45.907 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T03:17:45.907 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:45.907 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:45.907 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:45.907 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T03:17:45.907 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:45.908 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:46.122 INFO:teuthology.orchestra.run.smithi177.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 384 kB/s | 90 kB 00:00 2024-05-08T03:17:46.147 INFO:teuthology.orchestra.run.smithi177.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 244 kB/s | 206 kB 00:00 2024-05-08T03:17:46.153 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T03:17:46.757 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:27 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:46.884 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 142 k 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T03:17:46.885 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout:Install 6 Packages 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:46.886 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 7.0 M 2024-05-08T03:17:46.887 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 33 M 2024-05-08T03:17:46.887 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:46.998 INFO:teuthology.orchestra.run.smithi177.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 738 kB/s | 1.0 MB 00:01 2024-05-08T03:17:47.002 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:47.002 INFO:teuthology.orchestra.run.smithi177.stdout:Total 888 kB/s | 2.2 MB 00:02 2024-05-08T03:17:47.075 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:47.090 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:47.090 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:47.176 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:47.177 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:47.243 INFO:teuthology.orchestra.run.smithi188.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2529.ge0c1691a.e 680 kB/s | 142 kB 00:00 2024-05-08T03:17:47.560 INFO:teuthology.orchestra.run.smithi188.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.7 MB/s | 548 kB 00:00 2024-05-08T03:17:47.618 INFO:teuthology.orchestra.run.smithi188.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 990 kB/s | 58 kB 00:00 2024-05-08T03:17:47.677 INFO:teuthology.orchestra.run.smithi188.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 874 kB/s | 51 kB 00:00 2024-05-08T03:17:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:47.853 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T03:17:47.885 INFO:teuthology.orchestra.run.smithi188.stdout:(5/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 291 kB/s | 248 kB 00:00 2024-05-08T03:17:47.907 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T03:17:47.962 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T03:17:48.008 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T03:17:48.050 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T03:17:48.142 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T03:17:48.244 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T03:17:48.293 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T03:17:48.352 INFO:teuthology.orchestra.run.smithi188.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.5 MB/s | 6.0 MB 00:01 2024-05-08T03:17:48.353 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:48.353 INFO:teuthology.orchestra.run.smithi188.stdout:Total 4.8 MB/s | 7.0 MB 00:01 2024-05-08T03:17:48.424 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:48.436 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:48.437 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:48.533 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:48.533 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:48.846 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:49.000 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T03:17:49.017 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T03:17:49.048 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T03:17:49.070 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T03:17:49.082 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T03:17:49.129 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T03:17:49.151 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 13/13 2024-05-08T03:17:49.584 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T03:17:49.665 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T03:17:49.728 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T03:17:49.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 6/6 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 13/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 1/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T03:17:49.966 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T03:17:49.967 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T03:17:49.967 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T03:17:49.967 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T03:17:49.967 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T03:17:49.967 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T03:17:50.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:50.292 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 6/6 2024-05-08T03:17:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/6 2024-05-08T03:17:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T03:17:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T03:17:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T03:17:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T03:17:50.560 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T03:17:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:51.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-fuse 2024-05-08T03:17:51.119 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:29 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:51.241 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:51.242 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:51.242 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:51.242 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:51.242 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:51.242 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 142 k 2024-05-08T03:17:51.242 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout:Install 6 Packages 2024-05-08T03:17:51.243 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:51.244 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 7.0 M 2024-05-08T03:17:51.244 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 33 M 2024-05-08T03:17:51.244 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:51.599 INFO:teuthology.orchestra.run.smithi177.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-05-08T03:17:51.659 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:32 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:51.716 INFO:teuthology.orchestra.run.smithi177.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.6 MB/s | 548 kB 00:00 2024-05-08T03:17:51.750 INFO:teuthology.orchestra.run.smithi177.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-05-08T03:17:51.784 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:51.785 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:51.785 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:51.785 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:51.785 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 890 k 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:Install 2 Packages 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 970 k 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 2.8 M 2024-05-08T03:17:51.786 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:51.825 INFO:teuthology.orchestra.run.smithi177.stdout:(4/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-08T03:17:51.850 INFO:teuthology.orchestra.run.smithi177.stdout:(5/6): ceph-mgr-cephadm-19.0.0-2529.ge0c1691a.e 338 kB/s | 142 kB 00:00 2024-05-08T03:17:51.875 INFO:teuthology.orchestra.run.smithi177.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 407 kB/s | 51 kB 00:00 2024-05-08T03:17:51.876 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:51.876 INFO:teuthology.orchestra.run.smithi177.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-05-08T03:17:51.947 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:51.958 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:51.958 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:52.054 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:52.055 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:52.095 INFO:teuthology.orchestra.run.smithi188.stdout:(1/2): ceph-fuse-19.0.0-2529.ge0c1691a.el9.x86_ 3.2 MB/s | 890 kB 00:00 2024-05-08T03:17:52.246 INFO:teuthology.orchestra.run.smithi188.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 190 kB/s | 80 kB 00:00 2024-05-08T03:17:52.246 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:52.246 INFO:teuthology.orchestra.run.smithi188.stdout:Total 2.1 MB/s | 970 kB 00:00 2024-05-08T03:17:52.259 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:52.280 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:52.280 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:52.358 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:52.359 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:52.382 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:52.540 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T03:17:52.557 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:52.586 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T03:17:52.689 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T03:17:52.754 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/2 2024-05-08T03:17:53.095 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T03:17:53.179 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T03:17:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T03:17:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 6/6 2024-05-08T03:17:53.387 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/2 2024-05-08T03:17:53.387 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/2 2024-05-08T03:17:53.671 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T03:17:53.671 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:53.671 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:53.671 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T03:17:53.672 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:53.672 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:53.956 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-volume 2024-05-08T03:17:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 6/6 2024-05-08T03:17:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 1/6 2024-05-08T03:17:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T03:17:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T03:17:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T03:17:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T03:17:54.289 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:54.290 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:54.534 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:35 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:54.576 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-fuse 2024-05-08T03:17:54.661 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:54.661 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:54.661 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 264 k 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout:Install 2 Packages 2024-05-08T03:17:54.662 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:54.663 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 341 k 2024-05-08T03:17:54.663 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 1.5 M 2024-05-08T03:17:54.663 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:54.950 INFO:teuthology.orchestra.run.smithi188.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 542 kB/s | 77 kB 00:00 2024-05-08T03:17:55.158 INFO:teuthology.orchestra.run.smithi188.stdout:(2/2): ceph-volume-19.0.0-2529.ge0c1691a.el9.no 751 kB/s | 264 kB 00:00 2024-05-08T03:17:55.159 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:33 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:55.159 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:55.159 INFO:teuthology.orchestra.run.smithi188.stdout:Total 686 kB/s | 341 kB 00:00 2024-05-08T03:17:55.171 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:55.181 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:55.181 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:55.215 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:55.215 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:55.280 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 890 k 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T03:17:55.281 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:Install 2 Packages 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 970 k 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 2.8 M 2024-05-08T03:17:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:55.356 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:55.502 INFO:teuthology.orchestra.run.smithi177.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 437 kB/s | 80 kB 00:00 2024-05-08T03:17:55.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T03:17:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 2/2 2024-05-08T03:17:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 2/2 2024-05-08T03:17:55.593 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:55.593 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T03:17:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:55.861 INFO:teuthology.orchestra.run.smithi177.stdout:(2/2): ceph-fuse-19.0.0-2529.ge0c1691a.el9.x86_ 1.6 MB/s | 890 kB 00:00 2024-05-08T03:17:55.862 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:55.862 INFO:teuthology.orchestra.run.smithi177.stdout:Total 1.6 MB/s | 970 kB 00:00 2024-05-08T03:17:55.874 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:55.896 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:55.896 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:55.973 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:55.973 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:56.160 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:56.260 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 1/2 2024-05-08T03:17:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T03:17:56.394 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/2 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:56.570 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:56.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install librados-devel 2024-05-08T03:17:57.091 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2/2 2024-05-08T03:17:57.091 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/2 2024-05-08T03:17:57.374 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T03:17:57.375 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:57.375 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:17:57.375 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T03:17:57.375 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:57.375 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:17:57.480 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:38 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:57.604 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repo Size 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 127 k 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:17:57.605 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:57.606 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 127 k 2024-05-08T03:17:57.606 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 456 k 2024-05-08T03:17:57.606 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:17:57.667 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-volume 2024-05-08T03:17:57.921 INFO:teuthology.orchestra.run.smithi188.stdout:librados-devel-19.0.0-2529.ge0c1691a.el9.x86_64 403 kB/s | 127 kB 00:00 2024-05-08T03:17:57.922 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:57.922 INFO:teuthology.orchestra.run.smithi188.stdout:Total 400 kB/s | 127 kB 00:00 2024-05-08T03:17:57.922 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:17:57.927 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:17:57.928 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:17:57.995 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:17:57.995 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:17:58.157 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:17:58.250 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:36 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:17:58.272 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.0.0-2529.ge0c1691a.el9 ceph-noarch 264 k 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-05-08T03:17:58.376 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T03:17:58.377 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:58.377 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:17:58.377 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:17:58.377 INFO:teuthology.orchestra.run.smithi177.stdout:Install 2 Packages 2024-05-08T03:17:58.377 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:58.377 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 341 k 2024-05-08T03:17:58.378 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 1.5 M 2024-05-08T03:17:58.378 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:17:58.607 INFO:teuthology.orchestra.run.smithi177.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 477 kB/s | 77 kB 00:00 2024-05-08T03:17:58.690 INFO:teuthology.orchestra.run.smithi177.stdout:(2/2): ceph-volume-19.0.0-2529.ge0c1691a.el9.no 1.0 MB/s | 264 kB 00:00 2024-05-08T03:17:58.691 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:17:58.691 INFO:teuthology.orchestra.run.smithi177.stdout:Total 1.1 MB/s | 341 kB 00:00 2024-05-08T03:17:58.703 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:17:58.714 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:17:58.714 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:17:58.723 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:17:58.749 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:17:58.749 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:17:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:17:58.978 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:17:58.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:58.978 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:17:58.978 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:17:58.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:17:58.978 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:17:59.073 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T03:17:59.110 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 2/2 2024-05-08T03:17:59.126 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 2/2 2024-05-08T03:17:59.126 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:17:59.126 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T03:17:59.126 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:17:59.252 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install libcephfs2 2024-05-08T03:17:59.835 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:40 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:17:59.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 1/2 2024-05-08T03:17:59.918 INFO:teuthology.orchestra.run.smithi188.stdout:Package libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:17:59.963 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:17:59.964 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:17:59.964 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:00.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install libcephfs-devel 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.0.0-2529.ge0c1691a.el9.noarch 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:00.151 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:00.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados-devel 2024-05-08T03:18:00.594 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:41 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:00.717 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:00.718 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:00.718 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repo Size 2024-05-08T03:18:00.718 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 32 k 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 32 k 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 142 k 2024-05-08T03:18:00.719 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:18:00.962 INFO:teuthology.orchestra.run.smithi188.stdout:libcephfs-devel-19.0.0-2529.ge0c1691a.el9.x86_6 130 kB/s | 32 kB 00:00 2024-05-08T03:18:00.963 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:00.963 INFO:teuthology.orchestra.run.smithi188.stdout:Total 130 kB/s | 32 kB 00:00 2024-05-08T03:18:00.963 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:18:00.970 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:18:00.970 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:18:00.984 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:18:00.984 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:18:01.032 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:39 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:01.068 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:18:01.160 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 127 k 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:18:01.161 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:01.162 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 127 k 2024-05-08T03:18:01.162 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 456 k 2024-05-08T03:18:01.162 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:18:01.184 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:01.546 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:01.571 INFO:teuthology.orchestra.run.smithi177.stdout:librados-devel-19.0.0-2529.ge0c1691a.el9.x86_64 309 kB/s | 127 kB 00:00 2024-05-08T03:18:01.572 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:01.572 INFO:teuthology.orchestra.run.smithi177.stdout:Total 308 kB/s | 127 kB 00:00 2024-05-08T03:18:01.572 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:18:01.578 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:18:01.578 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:18:01.645 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:18:01.646 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:18:01.779 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:01.779 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:01.779 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:18:01.779 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:01.779 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:01.779 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:01.798 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:18:01.894 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:01.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install librados2 2024-05-08T03:18:02.316 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:02.543 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:43 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:02.564 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:02.565 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:02.565 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:18:02.565 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:02.565 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:02.565 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:02.626 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:02.667 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:02.668 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:18:02.669 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:02.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install librbd1 2024-05-08T03:18:02.823 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install libcephfs2 2024-05-08T03:18:03.295 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:03.379 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:03.406 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:41 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:03.423 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:03.423 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:18:03.423 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:03.487 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install python3-rados 2024-05-08T03:18:03.489 INFO:teuthology.orchestra.run.smithi177.stdout:Package libcephfs2-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:03.532 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:03.533 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:03.533 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:03.603 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install libcephfs-devel 2024-05-08T03:18:04.052 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:04.132 INFO:teuthology.orchestra.run.smithi188.stdout:Package python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:18:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:04.182 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:42 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:04.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install python3-rgw 2024-05-08T03:18:04.307 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:04.307 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 32 k 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 32 k 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 142 k 2024-05-08T03:18:04.308 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:18:04.816 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:04.897 INFO:teuthology.orchestra.run.smithi188.stdout:Package python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:04.936 INFO:teuthology.orchestra.run.smithi177.stdout:libcephfs-devel-19.0.0-2529.ge0c1691a.el9.x86_6 51 kB/s | 32 kB 00:00 2024-05-08T03:18:04.936 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:04.937 INFO:teuthology.orchestra.run.smithi177.stdout:Total 50 kB/s | 32 kB 00:00 2024-05-08T03:18:04.937 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:18:04.938 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:04.939 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:18:04.939 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:04.940 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:18:04.940 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:18:04.956 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:18:04.957 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:18:05.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install python3-cephfs 2024-05-08T03:18:05.041 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:18:05.150 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:05.501 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:05.571 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:46 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:05.653 INFO:teuthology.orchestra.run.smithi188.stdout:Package python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:05.696 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:05.696 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:18:05.697 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:05.728 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:05.729 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:05.729 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:18:05.729 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:05.729 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:05.729 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:05.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install python3-rbd 2024-05-08T03:18:05.917 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librados2 2024-05-08T03:18:06.332 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:06.413 INFO:teuthology.orchestra.run.smithi188.stdout:Package python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:06.455 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:06.456 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-05-08T03:18:06.456 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:06.492 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:06.521 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install rbd-fuse 2024-05-08T03:18:06.573 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:06.614 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:06.615 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:06.615 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:06.677 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install librbd1 2024-05-08T03:18:07.092 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:07.214 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:07.215 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:07.215 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-05-08T03:18:07.215 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:07.215 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:18:07.215 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 85 k 2024-05-08T03:18:07.215 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 85 k 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 231 k 2024-05-08T03:18:07.216 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:18:07.244 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:07.324 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:07.365 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:07.366 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:07.366 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:07.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rados 2024-05-08T03:18:07.732 INFO:teuthology.orchestra.run.smithi188.stdout:rbd-fuse-19.0.0-2529.ge0c1691a.el9.x86_64.rpm 165 kB/s | 85 kB 00:00 2024-05-08T03:18:07.732 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:07.732 INFO:teuthology.orchestra.run.smithi188.stdout:Total 165 kB/s | 85 kB 00:00 2024-05-08T03:18:07.733 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:18:07.740 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:18:07.740 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:18:07.803 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:18:07.803 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:18:07.947 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:18:07.999 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:08.072 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:08.080 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rados-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:08.122 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:08.123 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:08.123 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:08.184 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rgw 2024-05-08T03:18:08.461 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:08.721 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:08.721 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:08.722 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:18:08.722 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:08.722 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:08.722 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:08.756 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:46 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:08.837 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rgw-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:08.879 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:08.881 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:08.881 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:08.954 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-cephfs 2024-05-08T03:18:09.003 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install rbd-mirror 2024-05-08T03:18:09.523 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:09.577 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:09.604 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-cephfs-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:09.646 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:09.647 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:09.647 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:09.702 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repo Size 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 3.1 M 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:18:09.703 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:09.704 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:18:09.704 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:09.704 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 3.1 M 2024-05-08T03:18:09.704 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 13 M 2024-05-08T03:18:09.704 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:18:09.713 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install python3-rbd 2024-05-08T03:18:10.292 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:10.385 INFO:teuthology.orchestra.run.smithi177.stdout:Package python3-rbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 is already installed. 2024-05-08T03:18:10.428 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:10.429 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-05-08T03:18:10.429 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:10.491 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-fuse 2024-05-08T03:18:11.063 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:11.186 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 85 k 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:11.187 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:18:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:11.188 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 85 k 2024-05-08T03:18:11.188 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 231 k 2024-05-08T03:18:11.188 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:18:11.427 INFO:teuthology.orchestra.run.smithi188.stdout:rbd-mirror-19.0.0-2529.ge0c1691a.el9.x86_64.rpm 1.8 MB/s | 3.1 MB 00:01 2024-05-08T03:18:11.428 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:11.428 INFO:teuthology.orchestra.run.smithi188.stdout:Total 1.8 MB/s | 3.1 MB 00:01 2024-05-08T03:18:11.428 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:18:11.439 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:18:11.439 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:18:11.522 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:18:11.523 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:18:11.572 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-fuse-19.0.0-2529.ge0c1691a.el9.x86_64.rpm 222 kB/s | 85 kB 00:00 2024-05-08T03:18:11.573 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:11.573 INFO:teuthology.orchestra.run.smithi177.stdout:Total 221 kB/s | 85 kB 00:00 2024-05-08T03:18:11.573 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:18:11.580 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:18:11.580 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:18:11.643 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:18:11.644 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:18:11.786 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:18:11.894 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:11.945 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:18:12.019 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:12.044 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:12.044 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:18:12.044 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T03:18:12.044 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T03:18:12.045 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T03:18:12.045 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:12.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:12.496 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:12.496 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:18:12.496 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:12.496 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:12.496 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:12.768 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-mirror 2024-05-08T03:18:13.098 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:13.098 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:13.098 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:18:13.098 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:13.098 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:13.098 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:13.350 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:13.380 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install rbd-nbd 2024-05-08T03:18:13.475 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:13.476 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:13.476 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repo Size 2024-05-08T03:18:13.476 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:13.476 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:18:13.476 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 3.1 M 2024-05-08T03:18:13.476 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 3.1 M 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 13 M 2024-05-08T03:18:13.477 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:18:13.959 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 03:16:19 AM UTC. 2024-05-08T03:18:14.081 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-08T03:18:14.082 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:14.082 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-05-08T03:18:14.082 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:14.082 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 170 k 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout:Install 1 Package 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 170 k 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 494 k 2024-05-08T03:18:14.083 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-08T03:18:14.623 INFO:teuthology.orchestra.run.smithi188.stdout:rbd-nbd-19.0.0-2529.ge0c1691a.el9.x86_64.rpm 316 kB/s | 170 kB 00:00 2024-05-08T03:18:14.623 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:14.624 INFO:teuthology.orchestra.run.smithi188.stdout:Total 315 kB/s | 170 kB 00:00 2024-05-08T03:18:14.624 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-08T03:18:14.631 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-08T03:18:14.631 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-08T03:18:14.693 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-mirror-19.0.0-2529.ge0c1691a.el9.x86_64.rpm 2.5 MB/s | 3.1 MB 00:01 2024-05-08T03:18:14.693 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:14.694 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-08T03:18:14.694 INFO:teuthology.orchestra.run.smithi177.stdout:Total 2.5 MB/s | 3.1 MB 00:01 2024-05-08T03:18:14.694 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:18:14.694 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-08T03:18:14.705 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:18:14.705 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:18:14.789 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:18:14.789 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:18:14.846 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-08T03:18:14.953 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:15.228 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:18:15.285 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:15.310 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:15.310 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T03:18:15.310 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T03:18:15.310 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T03:18:15.310 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T03:18:15.310 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:15.364 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:15.606 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:15.607 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:15.607 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-08T03:18:15.607 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:15.607 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:15.607 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-08T03:18:15.882 DEBUG:teuthology.parallel:result is None 2024-05-08T03:18:16.380 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:16.380 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:16.380 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:18:16.380 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:16.380 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:16.380 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:16.620 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install rbd-nbd 2024-05-08T03:18:17.199 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:55 ago on Wed 08 May 2024 03:16:22 AM UTC. 2024-05-08T03:18:17.331 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-05-08T03:18:17.331 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.0.0-2529.ge0c1691a.el9 ceph 170 k 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-05-08T03:18:17.332 INFO:teuthology.orchestra.run.smithi177.stdout:Install 1 Package 2024-05-08T03:18:17.333 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:17.333 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 170 k 2024-05-08T03:18:17.334 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 494 k 2024-05-08T03:18:17.334 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-05-08T03:18:17.538 INFO:teuthology.orchestra.run.smithi177.stdout:rbd-nbd-19.0.0-2529.ge0c1691a.el9.x86_64.rpm 832 kB/s | 170 kB 00:00 2024-05-08T03:18:17.539 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:18:17.539 INFO:teuthology.orchestra.run.smithi177.stdout:Total 824 kB/s | 170 kB 00:00 2024-05-08T03:18:17.539 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-05-08T03:18:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-05-08T03:18:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-05-08T03:18:17.614 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-05-08T03:18:17.615 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-05-08T03:18:17.774 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-05-08T03:18:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:18.298 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:18.553 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 1/1 2024-05-08T03:18:18.553 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:18.554 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-05-08T03:18:18.554 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.0.0-2529.ge0c1691a.el9.x86_64 2024-05-08T03:18:18.554 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:18.554 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-05-08T03:18:18.808 DEBUG:teuthology.parallel:result is None 2024-05-08T03:18:18.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0c1691aade07c0b78ccff9531f65d3c80e3e220 2024-05-08T03:18:18.955 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T03:18:18.977 INFO:teuthology.orchestra.run.smithi177.stdout:19.0.0-2529.ge0c1691a.el9 2024-05-08T03:18:18.977 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2529.ge0c1691a.el9 2024-05-08T03:18:18.978 INFO:teuthology.task.install:The correct ceph version 19.0.0-2529.ge0c1691a is installed. 2024-05-08T03:18:18.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0c1691aade07c0b78ccff9531f65d3c80e3e220 2024-05-08T03:18:19.204 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T03:18:19.230 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2529.ge0c1691a.el9 2024-05-08T03:18:19.230 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2529.ge0c1691a.el9 2024-05-08T03:18:19.231 INFO:teuthology.task.install:The correct ceph version 19.0.0-2529.ge0c1691a is installed. 2024-05-08T03:18:19.232 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T03:18:19.232 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:19.233 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T03:18:19.259 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:19.259 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T03:18:19.300 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T03:18:19.301 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:19.301 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T03:18:19.326 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T03:18:19.390 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:19.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T03:18:19.419 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T03:18:19.486 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T03:18:19.486 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:19.487 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T03:18:19.511 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T03:18:19.579 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:19.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T03:18:19.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T03:18:19.674 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T03:18:19.675 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:19.675 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T03:18:19.700 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T03:18:19.765 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:19.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T03:18:19.791 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T03:18:19.858 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T03:18:19.949 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T03:18:19.950 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2024-05-08T03:18:19.952 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-05-08T03:18:19.977 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T03:18:19.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T03:18:20.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T03:18:20.043 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T03:18:20.044 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T03:18:20.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T03:18:20.119 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T03:18:20.119 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T03:18:20.119 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770', 'branch': 'wip-dis-testing-squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7696770', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing-squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing-squid', 'sha1': '9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5306, 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220', 'sleep_before_teardown': 0, 'subset': '21/99', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci', 'suite_sha1': '9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2', 'targets': {'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKKATMtGJ8hGd4tN4Nwj5xJG88rH0kXPsX/mOHcxW25hNQ2Sx7+crb+EQKE1pXYZsoZ+kIhMW1BRyGZ3l5XIBg=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOf+Ru6NMxzSb2AsZvdFNe9lVVa4GsCEve2nWrlJni+zZblRQg/jvlGJ8Mi9xuhplZFAxXEbqMlUjn164M4S9QY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-07_23:02:08', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T03:18:20.120 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T03:18:20.147 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T03:18:20.164 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T03:18:20.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T03:18:20.228 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:20.228 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T03:18:20.281 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T03:18:20.282 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-05-08T03:18:20.338 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T03:18:20.338 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:20.338 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T03:18:20.338 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:20.338 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:20.338 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-05-08 03:18:15.567538932 +0000 2024-05-08T03:18:20.339 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-05-08 03:15:43.410853829 +0000 2024-05-08T03:18:20.339 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-05-08 03:15:43.410853829 +0000 2024-05-08T03:18:20.339 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-05-08 03:15:43.410853829 +0000 2024-05-08T03:18:20.339 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T03:18:20.400 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-05-08T03:18:20.401 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-05-08T03:18:20.401 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000233104 s, 2.2 MB/s 2024-05-08T03:18:20.402 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T03:18:20.458 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-05-08 03:18:15.567538932 +0000 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-05-08 03:15:43.757844308 +0000 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-05-08 03:15:43.757844308 +0000 2024-05-08T03:18:20.513 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-05-08 03:15:43.757844308 +0000 2024-05-08T03:18:20.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T03:18:20.576 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-05-08T03:18:20.576 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-05-08T03:18:20.576 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000251844 s, 2.0 MB/s 2024-05-08T03:18:20.577 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T03:18:20.634 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-05-08 03:18:15.568538905 +0000 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-05-08 03:15:44.110834622 +0000 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-05-08 03:15:44.110834622 +0000 2024-05-08T03:18:20.689 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-05-08 03:15:44.110834622 +0000 2024-05-08T03:18:20.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T03:18:20.751 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-05-08T03:18:20.751 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-05-08T03:18:20.752 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000187683 s, 2.7 MB/s 2024-05-08T03:18:20.753 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T03:18:20.809 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-05-08 03:18:15.568538905 +0000 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-05-08 03:15:44.456825128 +0000 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-05-08 03:15:44.456825128 +0000 2024-05-08T03:18:20.865 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-05-08 03:15:44.456825128 +0000 2024-05-08T03:18:20.866 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T03:18:20.928 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-05-08T03:18:20.928 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-05-08T03:18:20.928 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000190849 s, 2.7 MB/s 2024-05-08T03:18:20.929 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T03:18:20.986 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:20.986 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:20.986 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T03:18:21.005 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T03:18:21.005 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-05-08T03:18:21.062 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T03:18:21.062 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:21.062 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-08T03:18:21.062 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:21.063 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:21.063 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-08 03:18:12.291094056 +0000 2024-05-08T03:18:21.063 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-08 03:15:41.907273759 +0000 2024-05-08T03:18:21.063 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-08 03:15:41.907273759 +0000 2024-05-08T03:18:21.063 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-08 03:15:41.907273759 +0000 2024-05-08T03:18:21.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T03:18:21.127 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-08T03:18:21.127 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-08T03:18:21.127 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000215832 s, 2.4 MB/s 2024-05-08T03:18:21.129 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T03:18:21.186 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-08 03:18:12.291094056 +0000 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-08 03:15:42.247267732 +0000 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-08 03:15:42.247267732 +0000 2024-05-08T03:18:21.246 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-08 03:15:42.247267732 +0000 2024-05-08T03:18:21.247 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T03:18:21.310 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-08T03:18:21.310 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-08T03:18:21.311 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00021809 s, 2.3 MB/s 2024-05-08T03:18:21.312 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T03:18:21.368 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-08 03:18:12.292094038 +0000 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-08 03:15:42.590261652 +0000 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-08 03:15:42.590261652 +0000 2024-05-08T03:18:21.425 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-08 03:15:42.590261652 +0000 2024-05-08T03:18:21.425 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T03:18:21.488 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-08T03:18:21.488 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-08T03:18:21.488 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000229793 s, 2.2 MB/s 2024-05-08T03:18:21.490 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T03:18:21.547 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-05-08T03:18:21.602 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-08 03:18:12.292094038 +0000 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-08 03:15:42.924255732 +0000 2024-05-08T03:18:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-08 03:15:42.924255732 +0000 2024-05-08T03:18:21.604 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-08 03:15:42.924255732 +0000 2024-05-08T03:18:21.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T03:18:21.666 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-08T03:18:21.666 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-08T03:18:21.666 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000263096 s, 1.9 MB/s 2024-05-08T03:18:21.668 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T03:18:21.724 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:21.724 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T03:18:21.724 INFO:tasks.ceph:Generating config... 2024-05-08T03:18:21.726 INFO:tasks.ceph:[client] rbd cache = True 2024-05-08T03:18:21.727 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-05-08T03:18:21.727 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-08T03:18:21.727 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-05-08T03:18:21.727 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-08T03:18:21.727 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T03:18:21.727 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T03:18:21.728 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T03:18:21.728 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T03:18:21.728 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T03:18:21.728 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T03:18:21.728 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T03:18:21.728 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-05-08T03:18:21.728 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T03:18:21.729 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T03:18:21.730 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T03:18:21.730 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T03:18:21.730 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-08T03:18:21.730 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T03:18:21.730 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-08T03:18:21.730 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T03:18:21.731 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-08T03:18:21.731 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T03:18:21.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T03:18:21.785 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T03:18:21.788 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T03:18:21.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T03:18:21.858 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]'), ('mon.b', '172.21.15.188')] 2024-05-08T03:18:21.858 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790],172.21.15.188', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-08T03:18:21.861 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:21.862 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T03:18:21.914 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --add b 172.21.15.188 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:22.003 INFO:teuthology.orchestra.run.smithi177.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:22.003 INFO:teuthology.orchestra.run.smithi177.stdout:monmaptool: generated fsid a7e973ac-16c0-41c5-817e-e3e8f10aad7b 2024-05-08T03:18:22.003 INFO:teuthology.orchestra.run.smithi177.stdout:setting min_mon_release = quincy 2024-05-08T03:18:22.003 INFO:teuthology.orchestra.run.smithi177.stdout:epoch 0 2024-05-08T03:18:22.003 INFO:teuthology.orchestra.run.smithi177.stdout:fsid a7e973ac-16c0-41c5-817e-e3e8f10aad7b 2024-05-08T03:18:22.003 INFO:teuthology.orchestra.run.smithi177.stdout:last_changed 2024-05-08T03:18:22.002824+0000 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:created 2024-05-08T03:18:22.002824+0000 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:min_mon_release 17 (quincy) 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:election_strategy: 1 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:0: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.a 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2024-05-08T03:18:22.004 INFO:teuthology.orchestra.run.smithi177.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T03:18:22.005 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T03:18:22.020 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a7e973ac-16c0-41c5-817e-e3e8f10aad7b... 2024-05-08T03:18:22.021 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T03:18:22.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-05-08T03:18:22.103 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-05-08T03:18:22.104 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-05-08T03:18:22.105 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790],172.21.15.188" 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: mon client directed command retry = 5 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on pool no app = False 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: ms inject socket failures = 5000 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = a7e973ac-16c0-41c5-817e-e3e8f10aad7b 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.106 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-05-08T03:18:22.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = stupid 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-05-08T03:18:22.108 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: osd blocked scrub grace period = 3600 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: osd max markdown count = 1000 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: osd shutdown pgref assert = True 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-05-08T03:18:22.109 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-05-08T03:18:22.110 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.111 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: rbd cache = True 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout: rbd cache policy = writearound 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-05-08T03:18:22.112 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-05-08T03:18:22.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-05-08T03:18:22.114 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790],172.21.15.188" 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon client directed command retry = 5 2024-05-08T03:18:22.115 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on pool no app = False 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: ms inject socket failures = 5000 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = a7e973ac-16c0-41c5-817e-e3e8f10aad7b 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-05-08T03:18:22.116 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.117 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = stupid 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-05-08T03:18:22.118 INFO:teuthology.orchestra.run.smithi188.stdout: osd blocked scrub grace period = 3600 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: osd max markdown count = 1000 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: osd shutdown pgref assert = True 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.119 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.120 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: rbd cache = True 2024-05-08T03:18:22.121 INFO:teuthology.orchestra.run.smithi188.stdout: rbd cache policy = writearound 2024-05-08T03:18:22.122 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-05-08T03:18:22.122 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-05-08T03:18:22.122 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-05-08T03:18:22.123 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T03:18:22.123 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T03:18:22.217 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T03:18:22.218 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:22.218 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T03:18:22.234 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:22.234 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T03:18:22.288 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:18:22.288 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:22.288 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T03:18:22.289 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T03:18:22.362 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:22.362 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:22.418 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:18:22.418 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:22.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T03:18:22.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T03:18:22.453 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:22.453 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:22.509 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T03:18:22.509 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T03:18:22.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T03:18:22.560 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T03:18:22.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T03:18:22.624 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T03:18:22.626 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T03:18:22.626 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T03:18:22.626 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T03:18:22.667 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T03:18:22.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T03:18:22.720 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T03:18:22.733 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T03:18:22.733 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T03:18:22.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T03:18:22.757 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:22.757 INFO:tasks.ceph:role: osd.0 2024-05-08T03:18:22.757 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:18:22.757 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:22.830 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:23.337 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-05-08T03:18:23.339 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-05-08T03:18:23.339 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T03:18:23.471 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T03:18:23.505 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T03:18:23.570 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:23.571 INFO:tasks.ceph:role: osd.1 2024-05-08T03:18:23.571 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:18:23.571 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T03:18:23.642 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:23.642 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:23.642 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:23.642 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:23.643 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:23.643 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:23.643 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:23.643 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:23.643 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:23.643 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:24.095 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-05-08T03:18:24.097 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-05-08T03:18:24.097 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T03:18:24.197 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T03:18:24.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T03:18:24.293 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:24.293 INFO:tasks.ceph:role: osd.2 2024-05-08T03:18:24.293 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:18:24.293 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:24.365 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:24.846 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-05-08T03:18:24.848 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-05-08T03:18:24.849 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T03:18:24.985 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T03:18:25.019 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T03:18:25.086 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:25.086 INFO:tasks.ceph:role: osd.3 2024-05-08T03:18:25.086 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:18:25.086 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:25.160 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:25.654 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-05-08T03:18:25.656 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-05-08T03:18:25.656 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T03:18:25.804 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T03:18:25.838 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:25.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:25.943+0000 7fd1c4821640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T03:18:25.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:25.943+0000 7fd1c4821640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T03:18:25.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:25.943+0000 7fd1c4821640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T03:18:25.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:25.943+0000 7fd1c4821640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T03:18:25.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:25.943+0000 7fd1c4821640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T03:18:25.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:25.944+0000 7fd1c4821640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T03:18:28.801 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T03:18:28.829 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:28.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:28.914+0000 7fedca50a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T03:18:28.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:28.914+0000 7fedca50a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T03:18:28.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:28.914+0000 7fedca50a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T03:18:28.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:28.914+0000 7fedca50a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T03:18:28.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:28.914+0000 7fedca50a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T03:18:28.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:28.915+0000 7fedca50a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T03:18:31.772 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T03:18:31.802 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:31.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:31.891+0000 7f1b0381d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T03:18:31.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:31.892+0000 7f1b0381d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T03:18:31.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:31.892+0000 7f1b0381d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T03:18:31.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:31.892+0000 7f1b0381d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T03:18:31.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:31.892+0000 7f1b0381d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T03:18:31.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:31.892+0000 7f1b0381d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T03:18:34.738 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T03:18:34.765 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:34.850 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:34.849+0000 7f8163830640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T03:18:34.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:34.849+0000 7f8163830640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T03:18:34.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:34.849+0000 7f8163830640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T03:18:34.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:34.849+0000 7f8163830640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T03:18:34.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:34.849+0000 7f8163830640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T03:18:34.851 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:18:34.850+0000 7f8163830640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T03:18:37.696 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T03:18:37.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T03:18:37.755 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:37.755 INFO:tasks.ceph:role: osd.4 2024-05-08T03:18:37.755 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:18:37.756 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T03:18:37.833 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:37.834 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:38.331 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-08T03:18:38.333 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-08T03:18:38.334 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T03:18:38.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T03:18:38.493 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T03:18:38.562 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:38.562 INFO:tasks.ceph:role: osd.5 2024-05-08T03:18:38.562 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:18:38.562 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T03:18:38.635 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:38.636 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:39.097 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-08T03:18:39.100 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-08T03:18:39.100 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T03:18:39.191 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T03:18:39.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T03:18:39.285 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:39.286 INFO:tasks.ceph:role: osd.6 2024-05-08T03:18:39.286 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:18:39.286 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T03:18:39.359 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:39.360 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:39.853 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-08T03:18:39.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-08T03:18:39.855 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T03:18:39.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T03:18:40.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T03:18:40.075 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T03:18:40.075 INFO:tasks.ceph:role: osd.7 2024-05-08T03:18:40.075 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:18:40.075 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T03:18:40.147 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T03:18:40.670 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-08T03:18:40.672 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-08T03:18:40.672 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T03:18:40.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T03:18:40.831 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:40.931 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:40.929+0000 7f22d461f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T03:18:40.932 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:40.929+0000 7f22d461f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T03:18:40.932 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:40.929+0000 7f22d461f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T03:18:40.932 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:40.929+0000 7f22d461f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T03:18:40.932 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:40.929+0000 7f22d461f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T03:18:40.932 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:40.930+0000 7f22d461f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T03:18:43.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T03:18:43.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:43.896 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:43.894+0000 7fe9b39f6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T03:18:43.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:43.894+0000 7fe9b39f6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T03:18:43.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:43.894+0000 7fe9b39f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T03:18:43.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:43.894+0000 7fe9b39f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T03:18:43.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:43.894+0000 7fe9b39f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T03:18:43.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:43.895+0000 7fe9b39f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T03:18:46.743 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T03:18:46.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:46.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:46.861+0000 7f2f739f6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T03:18:46.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:46.861+0000 7f2f739f6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T03:18:46.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:46.861+0000 7f2f739f6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T03:18:46.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:46.861+0000 7f2f739f6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T03:18:46.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:46.861+0000 7f2f739f6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T03:18:46.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:46.862+0000 7f2f739f6640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T03:18:49.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T03:18:49.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:49.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:49.845+0000 7fa9d5211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T03:18:49.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:49.845+0000 7fa9d5211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T03:18:49.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:49.845+0000 7fa9d5211640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T03:18:49.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:49.845+0000 7fa9d5211640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T03:18:49.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:49.845+0000 7fa9d5211640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T03:18:49.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-08T03:18:49.846+0000 7fa9d5211640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T03:18:52.702 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T03:18:52.731 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T03:18:52.731 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:52.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T03:18:52.762 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:52.762 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T03:18:52.831 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:52.831 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T03:18:52.900 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:52.900 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T03:18:52.968 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:52.968 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T03:18:53.037 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:53.038 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T03:18:53.068 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:53.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T03:18:53.136 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:53.136 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T03:18:53.203 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:53.204 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T03:18:53.271 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:53.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T03:18:53.339 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:18:53.339 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T03:18:53.357 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:18:53.357 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T03:18:53.395 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T03:18:53.395 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T03:18:53.400 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T03:18:53.438 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2024-05-08T03:18:53.438 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQD+7jpm/Z5ZIRAAzIBGb3scP9AsyDbprxTvnw== 2024-05-08T03:18:53.438 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2024-05-08T03:18:53.438 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB7zpmYI1OOBAAsrnDCeTtMAvPDAhtiJsGXA== 2024-05-08T03:18:53.438 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAE7zpmKROPNhAAN4elT/+oNc6iPxcttdZxoA== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAH7zpmnZ44NRAAUP4x1iXJ/r3QfdZujE/lTQ== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAK7zpmJyalMhAAUWUmjAnRxq45B5ZIqpvkZw== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQD+7jpmmLAoJRAAq+BeNII9J3aaDejMORb8UA== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAQ7zpmdJd5NxAAcmRZvSmXJ3kr1SvkXY3S/A== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAT7zpmPk5mNRAAOZiJiOSOXUtBm//7lgjerg== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAW7zpm2b9wMxAAwHisqdzuFV2U0cyEc9YSVg== 2024-05-08T03:18:53.439 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2024-05-08T03:18:53.440 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAZ7zpmdR99MhAA2RdB+3ZAFjvkpbM1BM6tgg== 2024-05-08T03:18:53.440 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2024-05-08T03:18:53.440 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQD+7jpm5/HAJxAAj49W/Y04YwOXxvnEGELKRg== 2024-05-08T03:18:53.440 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2024-05-08T03:18:53.440 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQD+7jpmXkbuKhAAXqfNUuqBuB2jsMjQeddSRw== 2024-05-08T03:18:53.459 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD+7jpm/Z5ZIRAAzIBGb3scP9AsyDbprxTvnw== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAB7zpmYI1OOBAAsrnDCeTtMAvPDAhtiJsGXA== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAE7zpmKROPNhAAN4elT/+oNc6iPxcttdZxoA== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAH7zpmnZ44NRAAUP4x1iXJ/r3QfdZujE/lTQ== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAK7zpmJyalMhAAUWUmjAnRxq45B5ZIqpvkZw== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD+7jpmmLAoJRAAq+BeNII9J3aaDejMORb8UA== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAQ7zpmdJd5NxAAcmRZvSmXJ3kr1SvkXY3S/A== 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-05-08T03:18:53.460 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAT7zpmPk5mNRAAOZiJiOSOXUtBm//7lgjerg== 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAW7zpm2b9wMxAAwHisqdzuFV2U0cyEc9YSVg== 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAZ7zpmdR99MhAA2RdB+3ZAFjvkpbM1BM6tgg== 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD+7jpm5/HAJxAAj49W/Y04YwOXxvnEGELKRg== 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-05-08T03:18:53.461 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQD+7jpmXkbuKhAAXqfNUuqBuB2jsMjQeddSRw== 2024-05-08T03:18:53.462 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T03:18:53.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T03:18:53.554 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.607 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.660 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.712 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.764 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T03:18:53.766 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T03:18:53.817 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.870 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.873 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.923 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.975 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:53.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T03:18:54.028 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T03:18:54.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T03:18:54.084 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T03:18:54.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T03:18:54.139 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T03:18:54.139 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T03:18:54.164 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T03:18:54.259 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T03:18:54.283 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T03:18:54.348 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T03:18:54.444 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T03:18:54.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T03:18:54.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T03:18:54.591 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T03:18:54.616 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:54.618 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T03:18:54.671 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T03:18:54.671 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T03:18:54.672 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T03:18:54.674 INFO:tasks.ceph.mon.a:Started 2024-05-08T03:18:54.674 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T03:18:54.674 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T03:18:54.676 INFO:tasks.ceph.mon.c:Started 2024-05-08T03:18:54.676 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T03:18:54.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T03:18:54.713 INFO:tasks.ceph.mon.b:Started 2024-05-08T03:18:54.714 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T03:18:54.714 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T03:18:54.714 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T03:18:54.716 INFO:tasks.ceph.mgr.y:Started 2024-05-08T03:18:54.716 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T03:18:54.717 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T03:18:54.718 INFO:tasks.ceph.mgr.x:Started 2024-05-08T03:18:54.719 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T03:18:54.719 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T03:19:00.103 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:00.101+0000 7fa0e91b3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T03:19:00.255 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:00.254+0000 7fa0e91b3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T03:19:00.308 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:00.306+0000 7fa0e91b3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T03:19:00.564 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:00.562+0000 7fa0e91b3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T03:19:00.609 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:00.607+0000 7fa0e91b3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T03:19:00.895 INFO:teuthology.orchestra.run.smithi177.stderr:adjusted tunables profile to default 2024-05-08T03:19:00.907 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T03:19:00.907 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T03:19:00.908 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:19:00.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T03:19:00.932 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:19:00.932 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T03:19:00.999 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:19:00.999 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T03:19:01.064 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-05-08T03:19:01.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T03:19:01.129 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:19:01.129 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T03:19:01.157 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:19:01.158 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T03:19:01.193 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:01.192+0000 7fb16ee43040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T03:19:01.218 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:19:01.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T03:19:01.284 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-08T03:19:01.284 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T03:19:01.349 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 5bdb4f77-60d7-46da-a930-5450259e00b4 0 2024-05-08T03:19:01.363 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:01.362+0000 7fb16ee43040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T03:19:01.394 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.392+0000 7fa0e91b3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T03:19:01.419 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:01.418+0000 7fb16ee43040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T03:19:01.481 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.479+0000 7fa0e91b3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T03:19:01.523 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.521+0000 7fa0e91b3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T03:19:01.550 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-05-08T03:19:01.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9 1 2024-05-08T03:19:01.687 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:01.686+0000 7fb16ee43040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T03:19:01.689 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.688+0000 7fa0e91b3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T03:19:01.734 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:01.733+0000 7fb16ee43040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T03:19:01.746 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.744+0000 7fa0e91b3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T03:19:01.752 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-05-08T03:19:01.763 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new c712ddbe-4915-4670-a683-4acdd11404e3 2 2024-05-08T03:19:01.799 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.797+0000 7fa0e91b3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T03:19:01.910 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:01.909+0000 7fa0e91b3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T03:19:01.952 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-05-08T03:19:01.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 3e5c0168-06af-4e9a-8dd1-eb25895cb480 3 2024-05-08T03:19:02.151 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-05-08T03:19:02.162 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 6668ff75-4ff3-47a0-ba04-95ae6e233fe0 4 2024-05-08T03:19:02.291 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:02.289+0000 7fa0e91b3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T03:19:02.349 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-05-08T03:19:02.361 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 9e02c1b8-134f-43bf-917c-966ba4eac713 5 2024-05-08T03:19:02.527 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:02.526+0000 7fb16ee43040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T03:19:02.548 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-05-08T03:19:02.550 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:02.548+0000 7fa0e91b3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T03:19:02.559 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 38f1ad8a-9e66-4e02-bf65-0d93e046ad5b 6 2024-05-08T03:19:02.602 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:02.600+0000 7fa0e91b3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T03:19:02.621 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:02.619+0000 7fb16ee43040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T03:19:02.666 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:02.665+0000 7fb16ee43040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T03:19:02.717 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:02.716+0000 7fa0e91b3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T03:19:02.799 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-05-08T03:19:02.811 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new bd284e7d-47ec-489a-bd56-d5710e1fad27 7 2024-05-08T03:19:02.883 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:02.882+0000 7fb16ee43040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T03:19:02.938 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:02.937+0000 7fb16ee43040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T03:19:02.993 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:02.992+0000 7fb16ee43040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T03:19:02.997 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-05-08T03:19:03.008 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T03:19:03.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T03:19:03.009 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.007+0000 7fa0e91b3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T03:19:03.011 INFO:tasks.ceph.osd.0:Started 2024-05-08T03:19:03.011 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T03:19:03.011 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T03:19:03.013 INFO:tasks.ceph.osd.1:Started 2024-05-08T03:19:03.014 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T03:19:03.014 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T03:19:03.016 INFO:tasks.ceph.osd.2:Started 2024-05-08T03:19:03.016 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T03:19:03.016 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T03:19:03.018 INFO:tasks.ceph.osd.3:Started 2024-05-08T03:19:03.019 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T03:19:03.019 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T03:19:03.021 INFO:tasks.ceph.osd.4:Started 2024-05-08T03:19:03.021 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T03:19:03.021 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T03:19:03.023 INFO:tasks.ceph.osd.5:Started 2024-05-08T03:19:03.023 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T03:19:03.023 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T03:19:03.025 INFO:tasks.ceph.osd.6:Started 2024-05-08T03:19:03.025 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T03:19:03.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T03:19:03.028 INFO:tasks.ceph.osd.7:Started 2024-05-08T03:19:03.028 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T03:19:03.069 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.067+0000 7fa0e91b3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T03:19:03.141 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:03.140+0000 7fb16ee43040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T03:19:03.242 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:03.242 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":10,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:19:02.992690+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T03:19:02.992690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:19:03.253 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T03:19:03.254 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T03:19:03.267 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.266+0000 7fa0e91b3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T03:19:03.405 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.403+0000 7fa0e91b3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T03:19:03.454 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.452+0000 7fa0e91b3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T03:19:03.519 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:03.518+0000 7fb16ee43040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T03:19:03.523 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.522+0000 7fa0e91b3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T03:19:03.568 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:03.566+0000 7fa0e91b3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T03:19:03.620 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:03.619+0000 7f3d7c716640 -1 Falling back to public interface 2024-05-08T03:19:03.625 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:03.624+0000 7fd6e09a3640 -1 Falling back to public interface 2024-05-08T03:19:03.629 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:03.627+0000 7f9def60e640 -1 Falling back to public interface 2024-05-08T03:19:03.630 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:03.629+0000 7fe800a19640 -1 Falling back to public interface 2024-05-08T03:19:03.633 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:03.631+0000 7f760781f640 -1 Falling back to public interface 2024-05-08T03:19:03.637 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:03.635+0000 7fce71e1f640 -1 Falling back to public interface 2024-05-08T03:19:03.638 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:03.637+0000 7fbe76d18640 -1 Falling back to public interface 2024-05-08T03:19:03.648 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:03.647+0000 7f94d1217640 -1 Falling back to public interface 2024-05-08T03:19:03.777 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:03.776+0000 7fb16ee43040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T03:19:03.829 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:03.828+0000 7fb16ee43040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T03:19:03.946 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:03.945+0000 7fb16ee43040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T03:19:04.044 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:04.042+0000 7fa0e91b3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T03:19:04.060 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T03:19:04.167 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:04.165+0000 7fa0e91b3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T03:19:04.234 INFO:teuthology.misc.health.smithi177.stdout: 2024-05-08T03:19:04.234 INFO:teuthology.misc.health.smithi177.stdout:{"epoch":10,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:19:02.992690+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T03:19:02.992690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:19:04.239 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.238+0000 7fb16ee43040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T03:19:04.244 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T03:19:04.287 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.286+0000 7fb16ee43040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T03:19:04.444 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.443+0000 7fb16ee43040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T03:19:04.580 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.579+0000 7fb16ee43040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T03:19:04.628 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.627+0000 7fb16ee43040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T03:19:04.698 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.696+0000 7fb16ee43040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T03:19:04.744 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:04.743+0000 7fb16ee43040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T03:19:05.178 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:05.177+0000 7fa088bd0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T03:19:05.179 INFO:tasks.ceph.mgr.y.smithi177.stderr:2024-05-08T03:19:05.177+0000 7fa088bd0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T03:19:05.217 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:05.216+0000 7fb16ee43040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T03:19:05.334 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-08T03:19:05.333+0000 7fb16ee43040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T03:19:07.089 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:07.088+0000 7f9def60e640 -1 osd.2 0 log_to_monitors true 2024-05-08T03:19:07.102 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:07.101+0000 7f3d7c716640 -1 osd.4 0 log_to_monitors true 2024-05-08T03:19:07.145 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:07.144+0000 7f94d1217640 -1 osd.7 0 log_to_monitors true 2024-05-08T03:19:07.161 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:07.159+0000 7fe800a19640 -1 osd.5 0 log_to_monitors true 2024-05-08T03:19:07.172 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:07.170+0000 7f760781f640 -1 osd.0 0 log_to_monitors true 2024-05-08T03:19:07.172 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:07.171+0000 7fd6e09a3640 -1 osd.6 0 log_to_monitors true 2024-05-08T03:19:07.177 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:07.176+0000 7fce71e1f640 -1 osd.3 0 log_to_monitors true 2024-05-08T03:19:07.179 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:07.178+0000 7fbe76d18640 -1 osd.1 0 log_to_monitors true 2024-05-08T03:19:09.677 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:09.676+0000 7f76029ef640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T03:19:09.765 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:09.764+0000 7f75fb021640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:09.804 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:09.803+0000 7f9dea731640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T03:19:09.908 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:09.907+0000 7f9de2d63640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:10.357 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:10.356+0000 7f3d77e89640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T03:19:10.374 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:10.373+0000 7f3d704bb640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:10.408 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:10.407+0000 7f94cc2c5640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T03:19:10.422 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:10.421+0000 7fd6dc116640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T03:19:10.430 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:10.429+0000 7f94c48f7640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:10.439 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:10.438+0000 7fd6d4748640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:10.442 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:10.441+0000 7fe7fbbc4640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T03:19:10.456 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:10.455+0000 7fe7f41f6640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:10.617 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:10.616+0000 7fce6cf44640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T03:19:10.629 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:10.629+0000 7fce65576640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:10.652 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:10.651+0000 7fbe7248b640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T03:19:10.673 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:10.672+0000 7fbe6aabd640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:19:11.050 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T03:19:11.336 INFO:teuthology.misc.health.smithi177.stdout: 2024-05-08T03:19:11.336 INFO:teuthology.misc.health.smithi177.stdout:{"epoch":14,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:19:10.716373+0000","last_up_change":"2024-05-08T03:19:10.716373+0000","last_in_change":"2024-05-08T03:19:02.992690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6809","nonce":2599410006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6811","nonce":2599410006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6815","nonce":2599410006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6813","nonce":2599410006}]},"public_addr":"172.21.15.177:6809/2599410006","cluster_addr":"172.21.15.177:6811/2599410006","heartbeat_back_addr":"172.21.15.177:6815/2599410006","heartbeat_front_addr":"172.21.15.177:6813/2599410006","state":["exists","up"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6823","nonce":3471641477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6827","nonce":3471641477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6831","nonce":3471641477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6829","nonce":3471641477}]},"public_addr":"172.21.15.177:6823/3471641477","cluster_addr":"172.21.15.177:6827/3471641477","heartbeat_back_addr":"172.21.15.177:6831/3471641477","heartbeat_front_addr":"172.21.15.177:6829/3471641477","state":["exists","up"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6801","nonce":2694847262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6803","nonce":2694847262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6807","nonce":2694847262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6805","nonce":2694847262}]},"public_addr":"172.21.15.177:6801/2694847262","cluster_addr":"172.21.15.177:6803/2694847262","heartbeat_back_addr":"172.21.15.177:6807/2694847262","heartbeat_front_addr":"172.21.15.177:6805/2694847262","state":["exists","up"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6817","nonce":4093771454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6819","nonce":4093771454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6826","nonce":4093771454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6822","nonce":4093771454}]},"public_addr":"172.21.15.177:6817/4093771454","cluster_addr":"172.21.15.177:6819/4093771454","heartbeat_back_addr":"172.21.15.177:6826/4093771454","heartbeat_front_addr":"172.21.15.177:6822/4093771454","state":["exists","up"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6801","nonce":1585905529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6803","nonce":1585905529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6807","nonce":1585905529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6805","nonce":1585905529}]},"public_addr":"172.21.15.188:6801/1585905529","cluster_addr":"172.21.15.188:6803/1585905529","heartbeat_back_addr":"172.21.15.188:6807/1585905529","heartbeat_front_addr":"172.21.15.188:6805/1585905529","state":["exists","up"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6817","nonce":1643429991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6819","nonce":1643429991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6823","nonce":1643429991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6821","nonce":1643429991}]},"public_addr":"172.21.15.188:6817/1643429991","cluster_addr":"172.21.15.188:6819/1643429991","heartbeat_back_addr":"172.21.15.188:6823/1643429991","heartbeat_front_addr":"172.21.15.188:6821/1643429991","state":["exists","up"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6809","nonce":3787670496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6811","nonce":3787670496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6815","nonce":3787670496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6813","nonce":3787670496}]},"public_addr":"172.21.15.188:6809/3787670496","cluster_addr":"172.21.15.188:6811/3787670496","heartbeat_back_addr":"172.21.15.188:6815/3787670496","heartbeat_front_addr":"172.21.15.188:6813/3787670496","state":["exists","up"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6825","nonce":405203122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6827","nonce":405203122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6831","nonce":405203122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6829","nonce":405203122}]},"public_addr":"172.21.15.188:6825/405203122","cluster_addr":"172.21.15.188:6827/405203122","heartbeat_back_addr":"172.21.15.188:6831/405203122","heartbeat_front_addr":"172.21.15.188:6829/405203122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:19:11.348 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T03:19:11.349 INFO:tasks.ceph:Creating RBD pool 2024-05-08T03:19:11.349 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T03:19:11.720 INFO:teuthology.orchestra.run.smithi177.stderr:pool 'rbd' created 2024-05-08T03:19:11.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T03:19:12.728 INFO:teuthology.orchestra.run.smithi177.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T03:19:12.742 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T03:19:12.743 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T03:19:12.743 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T03:19:12.743 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T03:19:12.744 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T03:19:13.122 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:13.133 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1936885337},{"type":"v1","addr":"172.21.15.177:6833","nonce":1936885337}]},"active_addr":"172.21.15.177:6833/1936885337","active_change":"2024-05-08T03:19:04.167562+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2600038353}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3444697585}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3339415030}]}]} 2024-05-08T03:19:13.140 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T03:19:13.140 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T03:19:13.140 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T03:19:13.429 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:13.429 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":16,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:19:12.719523+0000","last_up_change":"2024-05-08T03:19:10.716373+0000","last_in_change":"2024-05-08T03:19:02.992690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T03:19:11.648360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T03:19:12.206654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6809","nonce":2599410006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6811","nonce":2599410006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6815","nonce":2599410006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6813","nonce":2599410006}]},"public_addr":"172.21.15.177:6809/2599410006","cluster_addr":"172.21.15.177:6811/2599410006","heartbeat_back_addr":"172.21.15.177:6815/2599410006","heartbeat_front_addr":"172.21.15.177:6813/2599410006","state":["exists","up"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6823","nonce":3471641477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6827","nonce":3471641477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6831","nonce":3471641477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6829","nonce":3471641477}]},"public_addr":"172.21.15.177:6823/3471641477","cluster_addr":"172.21.15.177:6827/3471641477","heartbeat_back_addr":"172.21.15.177:6831/3471641477","heartbeat_front_addr":"172.21.15.177:6829/3471641477","state":["exists","up"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6801","nonce":2694847262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6803","nonce":2694847262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6807","nonce":2694847262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6805","nonce":2694847262}]},"public_addr":"172.21.15.177:6801/2694847262","cluster_addr":"172.21.15.177:6803/2694847262","heartbeat_back_addr":"172.21.15.177:6807/2694847262","heartbeat_front_addr":"172.21.15.177:6805/2694847262","state":["exists","up"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6817","nonce":4093771454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6819","nonce":4093771454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6826","nonce":4093771454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6822","nonce":4093771454}]},"public_addr":"172.21.15.177:6817/4093771454","cluster_addr":"172.21.15.177:6819/4093771454","heartbeat_back_addr":"172.21.15.177:6826/4093771454","heartbeat_front_addr":"172.21.15.177:6822/4093771454","state":["exists","up"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6801","nonce":1585905529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6803","nonce":1585905529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6807","nonce":1585905529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6805","nonce":1585905529}]},"public_addr":"172.21.15.188:6801/1585905529","cluster_addr":"172.21.15.188:6803/1585905529","heartbeat_back_addr":"172.21.15.188:6807/1585905529","heartbeat_front_addr":"172.21.15.188:6805/1585905529","state":["exists","up"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6817","nonce":1643429991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6819","nonce":1643429991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6823","nonce":1643429991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6821","nonce":1643429991}]},"public_addr":"172.21.15.188:6817/1643429991","cluster_addr":"172.21.15.188:6819/1643429991","heartbeat_back_addr":"172.21.15.188:6823/1643429991","heartbeat_front_addr":"172.21.15.188:6821/1643429991","state":["exists","up"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6809","nonce":3787670496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6811","nonce":3787670496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6815","nonce":3787670496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6813","nonce":3787670496}]},"public_addr":"172.21.15.188:6809/3787670496","cluster_addr":"172.21.15.188:6811/3787670496","heartbeat_back_addr":"172.21.15.188:6815/3787670496","heartbeat_front_addr":"172.21.15.188:6813/3787670496","state":["exists","up"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6825","nonce":405203122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6827","nonce":405203122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6831","nonce":405203122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6829","nonce":405203122}]},"public_addr":"172.21.15.188:6825/405203122","cluster_addr":"172.21.15.188:6827/405203122","heartbeat_back_addr":"172.21.15.188:6831/405203122","heartbeat_front_addr":"172.21.15.188:6829/405203122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.148758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.170231+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.129002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.179213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.075940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122695+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:10.421746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:19:13.442 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T03:19:13.442 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T03:19:13.732 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:13.732 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":17,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:19:13.724157+0000","last_up_change":"2024-05-08T03:19:10.716373+0000","last_in_change":"2024-05-08T03:19:02.992690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T03:19:11.648360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T03:19:12.206654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6809","nonce":2599410006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6811","nonce":2599410006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6815","nonce":2599410006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6813","nonce":2599410006}]},"public_addr":"172.21.15.177:6809/2599410006","cluster_addr":"172.21.15.177:6811/2599410006","heartbeat_back_addr":"172.21.15.177:6815/2599410006","heartbeat_front_addr":"172.21.15.177:6813/2599410006","state":["exists","up"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6823","nonce":3471641477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6827","nonce":3471641477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6831","nonce":3471641477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6829","nonce":3471641477}]},"public_addr":"172.21.15.177:6823/3471641477","cluster_addr":"172.21.15.177:6827/3471641477","heartbeat_back_addr":"172.21.15.177:6831/3471641477","heartbeat_front_addr":"172.21.15.177:6829/3471641477","state":["exists","up"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6801","nonce":2694847262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6803","nonce":2694847262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6807","nonce":2694847262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6805","nonce":2694847262}]},"public_addr":"172.21.15.177:6801/2694847262","cluster_addr":"172.21.15.177:6803/2694847262","heartbeat_back_addr":"172.21.15.177:6807/2694847262","heartbeat_front_addr":"172.21.15.177:6805/2694847262","state":["exists","up"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6817","nonce":4093771454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6819","nonce":4093771454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6826","nonce":4093771454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6822","nonce":4093771454}]},"public_addr":"172.21.15.177:6817/4093771454","cluster_addr":"172.21.15.177:6819/4093771454","heartbeat_back_addr":"172.21.15.177:6826/4093771454","heartbeat_front_addr":"172.21.15.177:6822/4093771454","state":["exists","up"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6801","nonce":1585905529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6803","nonce":1585905529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6807","nonce":1585905529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6805","nonce":1585905529}]},"public_addr":"172.21.15.188:6801/1585905529","cluster_addr":"172.21.15.188:6803/1585905529","heartbeat_back_addr":"172.21.15.188:6807/1585905529","heartbeat_front_addr":"172.21.15.188:6805/1585905529","state":["exists","up"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6817","nonce":1643429991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6819","nonce":1643429991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6823","nonce":1643429991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1643429991},{"type":"v1","addr":"172.21.15.188:6821","nonce":1643429991}]},"public_addr":"172.21.15.188:6817/1643429991","cluster_addr":"172.21.15.188:6819/1643429991","heartbeat_back_addr":"172.21.15.188:6823/1643429991","heartbeat_front_addr":"172.21.15.188:6821/1643429991","state":["exists","up"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6809","nonce":3787670496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6811","nonce":3787670496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6815","nonce":3787670496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6813","nonce":3787670496}]},"public_addr":"172.21.15.188:6809/3787670496","cluster_addr":"172.21.15.188:6811/3787670496","heartbeat_back_addr":"172.21.15.188:6815/3787670496","heartbeat_front_addr":"172.21.15.188:6813/3787670496","state":["exists","up"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6825","nonce":405203122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6827","nonce":405203122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6831","nonce":405203122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6829","nonce":405203122}]},"public_addr":"172.21.15.188:6825/405203122","cluster_addr":"172.21.15.188:6827/405203122","heartbeat_back_addr":"172.21.15.188:6831/405203122","heartbeat_front_addr":"172.21.15.188:6829/405203122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.148758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.170231+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.129002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.179213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.075940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122695+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:10.421746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:19:13.747 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T03:19:13.748 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T03:19:13.748 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T03:19:13.749 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T03:19:13.749 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T03:19:13.750 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T03:19:13.750 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T03:19:13.751 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T03:19:14.000 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.001 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.002 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T03:19:14.002 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T03:19:14.003 INFO:teuthology.orchestra.run.smithi177.stdout:60129542149 2024-05-08T03:19:14.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T03:19:14.009 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T03:19:14.010 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.011 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T03:19:14.016 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.016 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T03:19:14.023 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.023 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T03:19:14.130 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.131 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T03:19:14.600 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.615 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-05-08T03:19:14.621 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.625 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:14.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.6 2024-05-08T03:19:14.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.3 2024-05-08T03:19:14.641 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-05-08T03:19:14.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.1 2024-05-08T03:19:14.675 INFO:teuthology.orchestra.run.smithi177.stdout:60129542146 2024-05-08T03:19:14.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.7 2024-05-08T03:19:14.703 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-05-08T03:19:14.745 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:14.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-05-08T03:19:15.621 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T03:19:15.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T03:19:15.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T03:19:15.648 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T03:19:15.662 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T03:19:15.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T03:19:15.719 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T03:19:15.760 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T03:19:16.172 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:16.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-05-08T03:19:16.194 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.204 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:16.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.6 2024-05-08T03:19:16.219 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.232 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:16.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.3 2024-05-08T03:19:16.252 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.262 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:16.265 INFO:teuthology.orchestra.run.smithi177.stdout:60129542149 2024-05-08T03:19:16.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-05-08T03:19:16.280 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.1 2024-05-08T03:19:16.282 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.325 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:16.329 INFO:teuthology.orchestra.run.smithi177.stdout:60129542147 2024-05-08T03:19:16.336 INFO:teuthology.orchestra.run.smithi177.stdout:60129542148 2024-05-08T03:19:16.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2024-05-08T03:19:16.339 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.7 2024-05-08T03:19:16.344 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.5 2024-05-08T03:19:16.350 DEBUG:teuthology.parallel:result is None 2024-05-08T03:19:16.350 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T03:19:16.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:19:16.672 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:16.672 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:19:16.685 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-05-08T03:19:16.174505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217368,"kb_used_data":1184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757352,"statfs":{"total":773094113280,"available":772871528448,"internally_reserved":0,"allocated":1212416,"data_stored":353984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000371"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.732726+0000","last_change":"2024-05-08T03:19:12.731926+0000","last_active":"2024-05-08T03:19:12.732726+0000","last_peered":"2024-05-08T03:19:12.732726+0000","last_clean":"2024-05-08T03:19:12.732726+0000","last_became_active":"2024-05-08T03:19:12.731504+0000","last_became_peered":"2024-05-08T03:19:12.731504+0000","last_unstale":"2024-05-08T03:19:12.732726+0000","last_undegraded":"2024-05-08T03:19:12.732726+0000","last_fullsized":"2024-05-08T03:19:12.732726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:11.129997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.969189+0000","last_change":"2024-05-08T03:19:12.966376+0000","last_active":"2024-05-08T03:19:12.969189+0000","last_peered":"2024-05-08T03:19:12.969189+0000","last_clean":"2024-05-08T03:19:12.969189+0000","last_became_active":"2024-05-08T03:19:12.966006+0000","last_became_peered":"2024-05-08T03:19:12.966006+0000","last_unstale":"2024-05-08T03:19:12.969189+0000","last_undegraded":"2024-05-08T03:19:12.969189+0000","last_fullsized":"2024-05-08T03:19:12.969189+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:03:19.392446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.731916+0000","last_change":"2024-05-08T03:19:12.731258+0000","last_active":"2024-05-08T03:19:12.731916+0000","last_peered":"2024-05-08T03:19:12.731916+0000","last_clean":"2024-05-08T03:19:12.731916+0000","last_became_active":"2024-05-08T03:19:12.730838+0000","last_became_peered":"2024-05-08T03:19:12.730838+0000","last_unstale":"2024-05-08T03:19:12.731916+0000","last_undegraded":"2024-05-08T03:19:12.731916+0000","last_fullsized":"2024-05-08T03:19:12.731916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:55:43.824361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.730315+0000","last_change":"2024-05-08T03:19:12.732433+0000","last_active":"2024-05-08T03:19:13.730315+0000","last_peered":"2024-05-08T03:19:13.730315+0000","last_clean":"2024-05-08T03:19:13.730315+0000","last_became_active":"2024-05-08T03:19:12.732130+0000","last_became_peered":"2024-05-08T03:19:12.732130+0000","last_unstale":"2024-05-08T03:19:13.730315+0000","last_undegraded":"2024-05-08T03:19:13.730315+0000","last_fullsized":"2024-05-08T03:19:13.730315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:37.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'64","reported_seq":97,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.936317+0000","last_change":"2024-05-08T03:19:13.735020+0000","last_active":"2024-05-08T03:19:13.936317+0000","last_peered":"2024-05-08T03:19:13.936317+0000","last_clean":"2024-05-08T03:19:13.936317+0000","last_became_active":"2024-05-08T03:19:13.734713+0000","last_became_peered":"2024-05-08T03:19:13.734713+0000","last_unstale":"2024-05-08T03:19:13.936317+0000","last_undegraded":"2024-05-08T03:19:13.936317+0000","last_fullsized":"2024-05-08T03:19:13.936317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:22:04.891179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.729727+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:19:13.729727+0000","last_peered":"2024-05-08T03:19:13.729727+0000","last_clean":"2024-05-08T03:19:13.729727+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:19:13.729727+0000","last_undegraded":"2024-05-08T03:19:13.729727+0000","last_fullsized":"2024-05-08T03:19:13.729727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.732409+0000","last_change":"2024-05-08T03:19:12.966511+0000","last_active":"2024-05-08T03:19:13.732409+0000","last_peered":"2024-05-08T03:19:13.732409+0000","last_clean":"2024-05-08T03:19:13.732409+0000","last_became_active":"2024-05-08T03:19:12.965999+0000","last_became_peered":"2024-05-08T03:19:12.965999+0000","last_unstale":"2024-05-08T03:19:13.732409+0000","last_undegraded":"2024-05-08T03:19:13.732409+0000","last_fullsized":"2024-05-08T03:19:13.732409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:11.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.968929+0000","last_change":"2024-05-08T03:19:12.965962+0000","last_active":"2024-05-08T03:19:12.968929+0000","last_peered":"2024-05-08T03:19:12.968929+0000","last_clean":"2024-05-08T03:19:12.968929+0000","last_became_active":"2024-05-08T03:19:12.965629+0000","last_became_peered":"2024-05-08T03:19:12.965629+0000","last_unstale":"2024-05-08T03:19:12.968929+0000","last_undegraded":"2024-05-08T03:19:12.968929+0000","last_fullsized":"2024-05-08T03:19:12.968929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:58.005596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.732384+0000","last_change":"2024-05-08T03:19:12.966516+0000","last_active":"2024-05-08T03:19:13.732384+0000","last_peered":"2024-05-08T03:19:13.732384+0000","last_clean":"2024-05-08T03:19:13.732384+0000","last_became_active":"2024-05-08T03:19:12.966041+0000","last_became_peered":"2024-05-08T03:19:12.966041+0000","last_unstale":"2024-05-08T03:19:13.732384+0000","last_undegraded":"2024-05-08T03:19:13.732384+0000","last_fullsized":"2024-05-08T03:19:13.732384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:08.473548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:19:16.686 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:19:16.969 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:16.969 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:19:16.981 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-05-08T03:19:16.174505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217368,"kb_used_data":1184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757352,"statfs":{"total":773094113280,"available":772871528448,"internally_reserved":0,"allocated":1212416,"data_stored":353984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000371"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.732726+0000","last_change":"2024-05-08T03:19:12.731926+0000","last_active":"2024-05-08T03:19:12.732726+0000","last_peered":"2024-05-08T03:19:12.732726+0000","last_clean":"2024-05-08T03:19:12.732726+0000","last_became_active":"2024-05-08T03:19:12.731504+0000","last_became_peered":"2024-05-08T03:19:12.731504+0000","last_unstale":"2024-05-08T03:19:12.732726+0000","last_undegraded":"2024-05-08T03:19:12.732726+0000","last_fullsized":"2024-05-08T03:19:12.732726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:11.129997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.969189+0000","last_change":"2024-05-08T03:19:12.966376+0000","last_active":"2024-05-08T03:19:12.969189+0000","last_peered":"2024-05-08T03:19:12.969189+0000","last_clean":"2024-05-08T03:19:12.969189+0000","last_became_active":"2024-05-08T03:19:12.966006+0000","last_became_peered":"2024-05-08T03:19:12.966006+0000","last_unstale":"2024-05-08T03:19:12.969189+0000","last_undegraded":"2024-05-08T03:19:12.969189+0000","last_fullsized":"2024-05-08T03:19:12.969189+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:03:19.392446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.731916+0000","last_change":"2024-05-08T03:19:12.731258+0000","last_active":"2024-05-08T03:19:12.731916+0000","last_peered":"2024-05-08T03:19:12.731916+0000","last_clean":"2024-05-08T03:19:12.731916+0000","last_became_active":"2024-05-08T03:19:12.730838+0000","last_became_peered":"2024-05-08T03:19:12.730838+0000","last_unstale":"2024-05-08T03:19:12.731916+0000","last_undegraded":"2024-05-08T03:19:12.731916+0000","last_fullsized":"2024-05-08T03:19:12.731916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:55:43.824361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.730315+0000","last_change":"2024-05-08T03:19:12.732433+0000","last_active":"2024-05-08T03:19:13.730315+0000","last_peered":"2024-05-08T03:19:13.730315+0000","last_clean":"2024-05-08T03:19:13.730315+0000","last_became_active":"2024-05-08T03:19:12.732130+0000","last_became_peered":"2024-05-08T03:19:12.732130+0000","last_unstale":"2024-05-08T03:19:13.730315+0000","last_undegraded":"2024-05-08T03:19:13.730315+0000","last_fullsized":"2024-05-08T03:19:13.730315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:37.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'64","reported_seq":97,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.936317+0000","last_change":"2024-05-08T03:19:13.735020+0000","last_active":"2024-05-08T03:19:13.936317+0000","last_peered":"2024-05-08T03:19:13.936317+0000","last_clean":"2024-05-08T03:19:13.936317+0000","last_became_active":"2024-05-08T03:19:13.734713+0000","last_became_peered":"2024-05-08T03:19:13.734713+0000","last_unstale":"2024-05-08T03:19:13.936317+0000","last_undegraded":"2024-05-08T03:19:13.936317+0000","last_fullsized":"2024-05-08T03:19:13.936317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:22:04.891179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.729727+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:19:13.729727+0000","last_peered":"2024-05-08T03:19:13.729727+0000","last_clean":"2024-05-08T03:19:13.729727+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:19:13.729727+0000","last_undegraded":"2024-05-08T03:19:13.729727+0000","last_fullsized":"2024-05-08T03:19:13.729727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.732409+0000","last_change":"2024-05-08T03:19:12.966511+0000","last_active":"2024-05-08T03:19:13.732409+0000","last_peered":"2024-05-08T03:19:13.732409+0000","last_clean":"2024-05-08T03:19:13.732409+0000","last_became_active":"2024-05-08T03:19:12.965999+0000","last_became_peered":"2024-05-08T03:19:12.965999+0000","last_unstale":"2024-05-08T03:19:13.732409+0000","last_undegraded":"2024-05-08T03:19:13.732409+0000","last_fullsized":"2024-05-08T03:19:13.732409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:11.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.968929+0000","last_change":"2024-05-08T03:19:12.965962+0000","last_active":"2024-05-08T03:19:12.968929+0000","last_peered":"2024-05-08T03:19:12.968929+0000","last_clean":"2024-05-08T03:19:12.968929+0000","last_became_active":"2024-05-08T03:19:12.965629+0000","last_became_peered":"2024-05-08T03:19:12.965629+0000","last_unstale":"2024-05-08T03:19:12.968929+0000","last_undegraded":"2024-05-08T03:19:12.968929+0000","last_fullsized":"2024-05-08T03:19:12.968929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:58.005596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.732384+0000","last_change":"2024-05-08T03:19:12.966516+0000","last_active":"2024-05-08T03:19:13.732384+0000","last_peered":"2024-05-08T03:19:13.732384+0000","last_clean":"2024-05-08T03:19:13.732384+0000","last_became_active":"2024-05-08T03:19:12.966041+0000","last_became_peered":"2024-05-08T03:19:12.966041+0000","last_unstale":"2024-05-08T03:19:13.732384+0000","last_undegraded":"2024-05-08T03:19:13.732384+0000","last_fullsized":"2024-05-08T03:19:13.732384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:08.473548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:19:16.982 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T03:19:16.982 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T03:19:16.983 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T03:19:16.983 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T03:19:17.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:17.305 INFO:teuthology.orchestra.run.smithi177.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T03:19:17.318 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T03:19:17.318 INFO:teuthology.run_tasks:Running task thrashosds... 2024-05-08T03:19:17.330 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-05-08T03:19:17.330 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing-squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e0c1691aade07c0b78ccff9531f65d3c80e3e220'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing-squid', 'sha1': '9aa7e81b5f50a61b09e3bd562e7d676fd72d84f2'}} 2024-05-08T03:19:17.331 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-05-08T03:19:17.331 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-05-08T03:19:17.633 INFO:tasks.thrashosds:Beginning thrashosds... 2024-05-08T03:19:17.633 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T03:19:17.633 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:19:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:17.917 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:19:17.929 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-05-08T03:19:16.174505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217368,"kb_used_data":1184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757352,"statfs":{"total":773094113280,"available":772871528448,"internally_reserved":0,"allocated":1212416,"data_stored":353984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000371"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.732726+0000","last_change":"2024-05-08T03:19:12.731926+0000","last_active":"2024-05-08T03:19:12.732726+0000","last_peered":"2024-05-08T03:19:12.732726+0000","last_clean":"2024-05-08T03:19:12.732726+0000","last_became_active":"2024-05-08T03:19:12.731504+0000","last_became_peered":"2024-05-08T03:19:12.731504+0000","last_unstale":"2024-05-08T03:19:12.732726+0000","last_undegraded":"2024-05-08T03:19:12.732726+0000","last_fullsized":"2024-05-08T03:19:12.732726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:11.129997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.969189+0000","last_change":"2024-05-08T03:19:12.966376+0000","last_active":"2024-05-08T03:19:12.969189+0000","last_peered":"2024-05-08T03:19:12.969189+0000","last_clean":"2024-05-08T03:19:12.969189+0000","last_became_active":"2024-05-08T03:19:12.966006+0000","last_became_peered":"2024-05-08T03:19:12.966006+0000","last_unstale":"2024-05-08T03:19:12.969189+0000","last_undegraded":"2024-05-08T03:19:12.969189+0000","last_fullsized":"2024-05-08T03:19:12.969189+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:03:19.392446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.731916+0000","last_change":"2024-05-08T03:19:12.731258+0000","last_active":"2024-05-08T03:19:12.731916+0000","last_peered":"2024-05-08T03:19:12.731916+0000","last_clean":"2024-05-08T03:19:12.731916+0000","last_became_active":"2024-05-08T03:19:12.730838+0000","last_became_peered":"2024-05-08T03:19:12.730838+0000","last_unstale":"2024-05-08T03:19:12.731916+0000","last_undegraded":"2024-05-08T03:19:12.731916+0000","last_fullsized":"2024-05-08T03:19:12.731916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:55:43.824361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.730315+0000","last_change":"2024-05-08T03:19:12.732433+0000","last_active":"2024-05-08T03:19:13.730315+0000","last_peered":"2024-05-08T03:19:13.730315+0000","last_clean":"2024-05-08T03:19:13.730315+0000","last_became_active":"2024-05-08T03:19:12.732130+0000","last_became_peered":"2024-05-08T03:19:12.732130+0000","last_unstale":"2024-05-08T03:19:13.730315+0000","last_undegraded":"2024-05-08T03:19:13.730315+0000","last_fullsized":"2024-05-08T03:19:13.730315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:37.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'64","reported_seq":97,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.936317+0000","last_change":"2024-05-08T03:19:13.735020+0000","last_active":"2024-05-08T03:19:13.936317+0000","last_peered":"2024-05-08T03:19:13.936317+0000","last_clean":"2024-05-08T03:19:13.936317+0000","last_became_active":"2024-05-08T03:19:13.734713+0000","last_became_peered":"2024-05-08T03:19:13.734713+0000","last_unstale":"2024-05-08T03:19:13.936317+0000","last_undegraded":"2024-05-08T03:19:13.936317+0000","last_fullsized":"2024-05-08T03:19:13.936317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:22:04.891179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.729727+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:19:13.729727+0000","last_peered":"2024-05-08T03:19:13.729727+0000","last_clean":"2024-05-08T03:19:13.729727+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:19:13.729727+0000","last_undegraded":"2024-05-08T03:19:13.729727+0000","last_fullsized":"2024-05-08T03:19:13.729727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.732409+0000","last_change":"2024-05-08T03:19:12.966511+0000","last_active":"2024-05-08T03:19:13.732409+0000","last_peered":"2024-05-08T03:19:13.732409+0000","last_clean":"2024-05-08T03:19:13.732409+0000","last_became_active":"2024-05-08T03:19:12.965999+0000","last_became_peered":"2024-05-08T03:19:12.965999+0000","last_unstale":"2024-05-08T03:19:13.732409+0000","last_undegraded":"2024-05-08T03:19:13.732409+0000","last_fullsized":"2024-05-08T03:19:13.732409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:11.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T03:19:12.968929+0000","last_change":"2024-05-08T03:19:12.965962+0000","last_active":"2024-05-08T03:19:12.968929+0000","last_peered":"2024-05-08T03:19:12.968929+0000","last_clean":"2024-05-08T03:19:12.968929+0000","last_became_active":"2024-05-08T03:19:12.965629+0000","last_became_peered":"2024-05-08T03:19:12.965629+0000","last_unstale":"2024-05-08T03:19:12.968929+0000","last_undegraded":"2024-05-08T03:19:12.968929+0000","last_fullsized":"2024-05-08T03:19:12.968929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:58.005596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.732384+0000","last_change":"2024-05-08T03:19:12.966516+0000","last_active":"2024-05-08T03:19:13.732384+0000","last_peered":"2024-05-08T03:19:13.732384+0000","last_clean":"2024-05-08T03:19:13.732384+0000","last_became_active":"2024-05-08T03:19:12.966041+0000","last_became_peered":"2024-05-08T03:19:12.966041+0000","last_unstale":"2024-05-08T03:19:13.732384+0000","last_undegraded":"2024-05-08T03:19:13.732384+0000","last_fullsized":"2024-05-08T03:19:13.732384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:08.473548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:19:17.930 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:19:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:18.216 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:19:18.228 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-08T03:19:18.174803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219356,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755364,"statfs":{"total":773094113280,"available":772869492736,"internally_reserved":0,"allocated":2535424,"data_stored":1601068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":60,"num_write_kb":1306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000669"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648309+0000","last_change":"2024-05-08T03:19:12.731926+0000","last_active":"2024-05-08T03:19:14.648309+0000","last_peered":"2024-05-08T03:19:14.648309+0000","last_clean":"2024-05-08T03:19:14.648309+0000","last_became_active":"2024-05-08T03:19:12.731504+0000","last_became_peered":"2024-05-08T03:19:12.731504+0000","last_unstale":"2024-05-08T03:19:14.648309+0000","last_undegraded":"2024-05-08T03:19:14.648309+0000","last_fullsized":"2024-05-08T03:19:14.648309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:06:07.983173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648323+0000","last_change":"2024-05-08T03:19:12.966376+0000","last_active":"2024-05-08T03:19:14.648323+0000","last_peered":"2024-05-08T03:19:14.648323+0000","last_clean":"2024-05-08T03:19:14.648323+0000","last_became_active":"2024-05-08T03:19:12.966006+0000","last_became_peered":"2024-05-08T03:19:12.966006+0000","last_unstale":"2024-05-08T03:19:14.648323+0000","last_undegraded":"2024-05-08T03:19:14.648323+0000","last_fullsized":"2024-05-08T03:19:14.648323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:19.151925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648911+0000","last_change":"2024-05-08T03:19:12.731258+0000","last_active":"2024-05-08T03:19:14.648911+0000","last_peered":"2024-05-08T03:19:14.648911+0000","last_clean":"2024-05-08T03:19:14.648911+0000","last_became_active":"2024-05-08T03:19:12.730838+0000","last_became_peered":"2024-05-08T03:19:12.730838+0000","last_unstale":"2024-05-08T03:19:14.648911+0000","last_undegraded":"2024-05-08T03:19:14.648911+0000","last_fullsized":"2024-05-08T03:19:14.648911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:09.285333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.730315+0000","last_change":"2024-05-08T03:19:12.732433+0000","last_active":"2024-05-08T03:19:13.730315+0000","last_peered":"2024-05-08T03:19:13.730315+0000","last_clean":"2024-05-08T03:19:13.730315+0000","last_became_active":"2024-05-08T03:19:12.732130+0000","last_became_peered":"2024-05-08T03:19:12.732130+0000","last_unstale":"2024-05-08T03:19:13.730315+0000","last_undegraded":"2024-05-08T03:19:13.730315+0000","last_fullsized":"2024-05-08T03:19:13.730315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:37.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":162,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:15.155757+0000","last_change":"2024-05-08T03:19:13.735020+0000","last_active":"2024-05-08T03:19:15.155757+0000","last_peered":"2024-05-08T03:19:15.155757+0000","last_clean":"2024-05-08T03:19:15.155757+0000","last_became_active":"2024-05-08T03:19:13.734713+0000","last_became_peered":"2024-05-08T03:19:13.734713+0000","last_unstale":"2024-05-08T03:19:15.155757+0000","last_undegraded":"2024-05-08T03:19:15.155757+0000","last_fullsized":"2024-05-08T03:19:15.155757+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:54.753213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.737014+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:19:14.737014+0000","last_peered":"2024-05-08T03:19:14.737014+0000","last_clean":"2024-05-08T03:19:14.737014+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:19:14.737014+0000","last_undegraded":"2024-05-08T03:19:14.737014+0000","last_fullsized":"2024-05-08T03:19:14.737014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.741329+0000","last_change":"2024-05-08T03:19:12.966511+0000","last_active":"2024-05-08T03:19:14.741329+0000","last_peered":"2024-05-08T03:19:14.741329+0000","last_clean":"2024-05-08T03:19:14.741329+0000","last_became_active":"2024-05-08T03:19:12.965999+0000","last_became_peered":"2024-05-08T03:19:12.965999+0000","last_unstale":"2024-05-08T03:19:14.741329+0000","last_undegraded":"2024-05-08T03:19:14.741329+0000","last_fullsized":"2024-05-08T03:19:14.741329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:11.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648095+0000","last_change":"2024-05-08T03:19:12.965962+0000","last_active":"2024-05-08T03:19:14.648095+0000","last_peered":"2024-05-08T03:19:14.648095+0000","last_clean":"2024-05-08T03:19:14.648095+0000","last_became_active":"2024-05-08T03:19:12.965629+0000","last_became_peered":"2024-05-08T03:19:12.965629+0000","last_unstale":"2024-05-08T03:19:14.648095+0000","last_undegraded":"2024-05-08T03:19:14.648095+0000","last_fullsized":"2024-05-08T03:19:14.648095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:50.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.741313+0000","last_change":"2024-05-08T03:19:12.966516+0000","last_active":"2024-05-08T03:19:14.741313+0000","last_peered":"2024-05-08T03:19:14.741313+0000","last_clean":"2024-05-08T03:19:14.741313+0000","last_became_active":"2024-05-08T03:19:12.966041+0000","last_became_peered":"2024-05-08T03:19:12.966041+0000","last_unstale":"2024-05-08T03:19:14.741313+0000","last_undegraded":"2024-05-08T03:19:14.741313+0000","last_fullsized":"2024-05-08T03:19:14.741313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:08.473548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":770048,"data_stored":646522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":56154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":770048,"data_stored":646522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:19:18.229 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T03:19:18.230 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-05-08T03:19:18.517 INFO:teuthology.orchestra.run.smithi177.stdout:epoch 18 2024-05-08T03:19:18.517 INFO:teuthology.orchestra.run.smithi177.stdout:fsid a7e973ac-16c0-41c5-817e-e3e8f10aad7b 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:created 2024-05-08T03:18:59.968487+0000 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:modified 2024-05-08T03:19:14.733912+0000 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:crush_version 5 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:full_ratio 0.95 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:backfillfull_ratio 0.9 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:nearfull_ratio 0.85 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:require_min_compat_client luminous 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:min_compat_client jewel 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:require_osd_release squid 2024-05-08T03:19:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:stretch_mode_enabled false 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd read_balance_score 2.00 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:max_osd 8 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/2599410006,v1:172.21.15.177:6809/2599410006] [v2:172.21.15.177:6810/2599410006,v1:172.21.15.177:6811/2599410006] exists,up 5bdb4f77-60d7-46da-a930-5450259e00b4 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6821/3471641477,v1:172.21.15.177:6823/3471641477] [v2:172.21.15.177:6825/3471641477,v1:172.21.15.177:6827/3471641477] exists,up 1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/2694847262,v1:172.21.15.177:6801/2694847262] [v2:172.21.15.177:6802/2694847262,v1:172.21.15.177:6803/2694847262] exists,up c712ddbe-4915-4670-a683-4acdd11404e3 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/4093771454,v1:172.21.15.177:6817/4093771454] [v2:172.21.15.177:6818/4093771454,v1:172.21.15.177:6819/4093771454] exists,up 3e5c0168-06af-4e9a-8dd1-eb25895cb480 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/1585905529,v1:172.21.15.188:6801/1585905529] [v2:172.21.15.188:6802/1585905529,v1:172.21.15.188:6803/1585905529] exists,up 6668ff75-4ff3-47a0-ba04-95ae6e233fe0 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/1643429991,v1:172.21.15.188:6817/1643429991] [v2:172.21.15.188:6818/1643429991,v1:172.21.15.188:6819/1643429991] exists,up 9e02c1b8-134f-43bf-917c-966ba4eac713 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/3787670496,v1:172.21.15.188:6809/3787670496] [v2:172.21.15.188:6810/3787670496,v1:172.21.15.188:6811/3787670496] exists,up 38f1ad8a-9e66-4e02-bf65-0d93e046ad5b 2024-05-08T03:19:18.519 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/405203122,v1:172.21.15.188:6825/405203122] [v2:172.21.15.188:6826/405203122,v1:172.21.15.188:6827/405203122] exists,up bd284e7d-47ec-489a-bd56-d5710e1fad27 2024-05-08T03:19:18.531 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/2599410006,v1:172.21.15.177:6809/2599410006] [v2:172.21.15.177:6810/2599410006,v1:172.21.15.177:6811/2599410006] exists,up 5bdb4f77-60d7-46da-a930-5450259e00b4', 'osd.1 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6821/3471641477,v1:172.21.15.177:6823/3471641477] [v2:172.21.15.177:6825/3471641477,v1:172.21.15.177:6827/3471641477] exists,up 1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/2694847262,v1:172.21.15.177:6801/2694847262] [v2:172.21.15.177:6802/2694847262,v1:172.21.15.177:6803/2694847262] exists,up c712ddbe-4915-4670-a683-4acdd11404e3', 'osd.3 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/4093771454,v1:172.21.15.177:6817/4093771454] [v2:172.21.15.177:6818/4093771454,v1:172.21.15.177:6819/4093771454] exists,up 3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'osd.4 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/1585905529,v1:172.21.15.188:6801/1585905529] [v2:172.21.15.188:6802/1585905529,v1:172.21.15.188:6803/1585905529] exists,up 6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'osd.5 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/1643429991,v1:172.21.15.188:6817/1643429991] [v2:172.21.15.188:6818/1643429991,v1:172.21.15.188:6819/1643429991] exists,up 9e02c1b8-134f-43bf-917c-966ba4eac713', 'osd.6 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/3787670496,v1:172.21.15.188:6809/3787670496] [v2:172.21.15.188:6810/3787670496,v1:172.21.15.188:6811/3787670496] exists,up 38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'osd.7 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/405203122,v1:172.21.15.188:6825/405203122] [v2:172.21.15.188:6826/405203122,v1:172.21.15.188:6827/405203122] exists,up bd284e7d-47ec-489a-bd56-d5710e1fad27'] 2024-05-08T03:19:18.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-05-08T03:19:18.632 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:18.632 INFO:teuthology.orchestra.run.smithi177.stdout: "name": "mon.a", 2024-05-08T03:19:18.632 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster": "ceph", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "admin_socket_mode": "", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_client_required": "cephx, none", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_cluster_required": "cephx", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_debug": "true", 2024-05-08T03:19:18.633 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_service_required": "cephx", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_service_ticket_ttl": "240.000000", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "auth_supported": "cephx", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_aio": "true", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_aio_max_queue_depth": "1024", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_aio_poll_ms": "250", 2024-05-08T03:19:18.634 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_aio_reap_max": "16", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_async_discard": "false", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_block_size": "4096", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_debug_aio": "false", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_debug_inflight_ios": "false", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_enable_discard": "false", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_flock_retry": "3", 2024-05-08T03:19:18.635 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_flock_retry_interval": "0.100000", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_inject_crash": "0", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_inject_crash_flush_delay": "2", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_ioring": "false", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_ioring_hipri": "false", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_ioring_sqthread_poll": "false", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_read_buffer_alignment": "4096", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bdev_type": "", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_alloc_size": "1048576", 2024-05-08T03:19:18.636 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_allocator": "hybrid", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_buffered_io": "true", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_check_for_zeros": "false", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_check_volume_selector_often": "false", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_compact_log_sync": "false", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_log_compact_min_size": "16777216", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_log_replay_check_allocations": "true", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_max_log_runway": "4194304", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_max_prefetch": "1048576", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_min_flush_size": "524288", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_min_log_runway": "1048576", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_replay_recovery": "false", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-05-08T03:19:18.637 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_shared_alloc_size": "65536", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_sync_write": "false", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_allocation_from_file": "true", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_allocator": "hybrid", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blobid_prealloc": "10240", 2024-05-08T03:19:18.638 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_create": "true", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_db_create": "false", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_db_path": "", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_db_size": "0", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_path": "", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_preallocate_file": "false", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_size": "107374182400", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_wal_create": "false", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_wal_path": "", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_block_wal_size": "100663296", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_bluefs": "true", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_bluefs_env_mirror": "false", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-05-08T03:19:18.639 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_autotune": "true", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_size": "0", 2024-05-08T03:19:18.640 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_type": "2q", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_clone_cow": "true", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_algorithm": "snappy", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_max_blob_size": "0", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_min_blob_size": "0", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_mode": "none", 2024-05-08T03:19:18.641 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_csum_type": "crc32c", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_enforce_settings": "default", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_freelist": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_fsck_abort": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_inject_read_err": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_legacy_omap": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_prefill": "0.000000", 2024-05-08T03:19:18.642 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_small_allocations": "0", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_default_buffered_read": "true", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_default_buffered_write": "false", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_deferred_batch_ops": "0", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_elastic_shared_blobs": "true", 2024-05-08T03:19:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fail_eio": "false", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-05-08T03:19:18.644 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_on_mkfs": "true", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_on_mount": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_on_umount": "false", 2024-05-08T03:19:18.645 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_ignore_data_csum": "false", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_kvbackend": "rocksdb", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_log_op_age": "5.000000", 2024-05-08T03:19:18.646 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_max_alloc_size": "0", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_max_blob_size": "0", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_max_defer_interval": "3.000000", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_max_deferred_txc": "32", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_min_alloc_size": "0", 2024-05-08T03:19:18.647 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_nid_prealloc": "1024", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_prefer_deferred_size": "0", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_qfsck_on_mount": "true", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_retry_disk_reads": "3", 2024-05-08T03:19:18.648 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_rocksdb_cf": "true", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_rocksdb_options_annex": "", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_spdk_coremask": "0x1", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_spdk_io_sleep": "5", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_spdk_max_io_completion": "0", 2024-05-08T03:19:18.649 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_spdk_mem": "512", 2024-05-08T03:19:18.650 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_sync_submit_transaction": "false", 2024-05-08T03:19:18.650 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_throttle_bytes": "67108864", 2024-05-08T03:19:18.650 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_throttle_cost_per_io": "0", 2024-05-08T03:19:18.650 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-05-08T03:19:18.650 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-05-08T03:19:18.650 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_tracing": "false", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_volume_selection_reserved": "0", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-05-08T03:19:18.651 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-05-08T03:19:18.652 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-05-08T03:19:18.652 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-05-08T03:19:18.652 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-05-08T03:19:18.652 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-05-08T03:19:18.652 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_zero_block_detection": "false", 2024-05-08T03:19:18.652 INFO:teuthology.orchestra.run.smithi177.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_action_update_interval": "2", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_mount_timeout": "10", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-05-08T03:19:18.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_cluster_require_signatures": "false", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_cluster_require_version": "2", 2024-05-08T03:19:18.654 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_require_signatures": "false", 2024-05-08T03:19:18.655 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_require_version": "2", 2024-05-08T03:19:18.655 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_service_require_signatures": "false", 2024-05-08T03:19:18.655 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_service_require_version": "2", 2024-05-08T03:19:18.655 INFO:teuthology.orchestra.run.smithi177.stdout: "cephx_sign_messages": "true", 2024-05-08T03:19:18.655 INFO:teuthology.orchestra.run.smithi177.stdout: "chdir": "", 2024-05-08T03:19:18.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client_acl_type": "", 2024-05-08T03:19:18.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_asio_thread_count": "2", 2024-05-08T03:19:18.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_cache_mid": "0.750000", 2024-05-08T03:19:18.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_cache_size": "16384", 2024-05-08T03:19:18.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_caps_release_delay": "5", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_check_pool_perm": "true", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_collect_and_send_global_metrics": "false", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_debug_force_sync_read": "false", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_debug_getattr_caps": "false", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_debug_inject_tick_delay": "0", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-05-08T03:19:18.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_die_on_failed_remount": "false", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_dirsize_rbytes": "true", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_force_lazyio": "false", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_fs": "", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_inject_fixed_oldest_tid": "false", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_inject_release_failure": "false", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_max_inline_size": "4096", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_max_retries_on_remount_failure": "5", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_mds_namespace": "", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_metadata": "", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_mount_gid": "-1", 2024-05-08T03:19:18.658 INFO:teuthology.orchestra.run.smithi177.stdout: "client_mount_timeout": "300", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_mount_uid": "-1", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_mountpoint": "/", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_notify_timeout": "10", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_oc": "true", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_oc_max_dirty": "104857600", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_oc_max_dirty_age": "5.000000", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_oc_max_objects": "1000", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_oc_size": "209715200", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_oc_target_dirty": "8388608", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_permissions": "true", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_quota": "true", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_quota_df": "true", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_readahead_max_bytes": "0", 2024-05-08T03:19:18.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_readahead_max_periods": "4", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_readahead_min": "131072", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_reconnect_stale": "false", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_shutdown_timeout": "30", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_snapdir": ".snap", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_tick_interval": "1", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_trace": "", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_try_dentry_invalidate": "false", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_use_faked_inos": "false", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_use_random_mds": "false", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_graylog": "false", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_graylog_port": "12201", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_monitors": "default=true", 2024-05-08T03:19:18.660 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_syslog": "false", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "clog_to_syslog_level": "info", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "-", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_network": "", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_network_interface": "", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "compressor_zlib_isal": "false", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "compressor_zlib_level": "5", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "compressor_zlib_winsize": "-15", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "compressor_zstd_level": "1", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "crimson_alien_op_num_threads": "6", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "crimson_alien_thread_cpu_cores": "", 2024-05-08T03:19:18.661 INFO:teuthology.orchestra.run.smithi177.stdout: "crimson_osd_obc_lru_size": "512", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "crimson_seastar_cpu_cores": "", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_location": "", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_location_hook": "", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_location_hook_timeout": "10", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "daemonize": "false", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "daos_pool": "tank", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "dbstore_db_name_prefix": "dbstore", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_alienstore": "0/5", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_allow_any_pool_priority": "false", 2024-05-08T03:19:18.662 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_asok": "1/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_asok_assert_abort": "false", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_asserts_on_shutdown": "true", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_auth": "20/20", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_bdev": "1/3", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_bluefs": "1/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_bluestore": "1/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_buffer": "0/1", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_ceph_exporter": "1/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_cephfs_mirror": "0/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_cephsqlite": "0/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_client": "0/5", 2024-05-08T03:19:18.663 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_compressor": "1/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_context": "0/1", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_crimson_interrupt": "0/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_crush": "1/1", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_crypto": "1/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_cyanstore": "0/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_deliberately_leak_memory": "false", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_disable_randomized_ping": "false", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_dpdk": "1/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_eventtrace": "1/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_filer": "0/1", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_filestore": "1/3", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_finisher": "1/1", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_fuse": "1/5", 2024-05-08T03:19:18.664 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_heartbeat_testing_span": "0", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_heartbeatmap": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_immutable_obj_cache": "0/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_javaclient": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_journal": "1/3", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_journaler": "0/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_kstore": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_lockdep": "0/1", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mclock": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds_balancer": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds_locker": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds_log": "1/5", 2024-05-08T03:19:18.665 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds_log_expire": "1/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds_migrator": "1/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mds_quiesce": "3/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_memstore": "1/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mgr": "2/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mgrc": "1/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_mon": "20/20", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_monc": "0/10", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_ms": "1/1", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_none": "0/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_objclass": "0/5", 2024-05-08T03:19:18.666 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_objectcacher": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_objecter": "0/1", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_optracker": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_osd": "1/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_paxos": "20/20", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_perfcounter": "1/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_prioritycache": "1/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rados": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rbd": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rbd_mirror": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rbd_pwl": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rbd_replay": "0/5", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_refs": "0/0", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_reserver": "1/1", 2024-05-08T03:19:18.667 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw_access": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw_datacache": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw_dbstore": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw_flight": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw_lifecycle": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rgw_sync": "1/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_rocksdb": "4/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_backref": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_cache": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_cleaner": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_device": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_epm": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-05-08T03:19:18.668 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_journal": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_lba": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_odata": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_omap": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_onode": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_t": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_seastore_tm": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_striper": "0/1", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_test": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_throttle": "1/1", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_timer": "0/1", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_tp": "0/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "debug_trace": "1/5", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-05-08T03:19:18.669 INFO:teuthology.orchestra.run.smithi177.stdout: "device_failure_prediction_mode": "none", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "err_to_graylog": "false", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "err_to_journald": "false", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "err_to_stderr": "true", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "err_to_syslog": "false", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event_tracing": "false", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "exporter_addr": "0.0.0.0", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "exporter_http_port": "9926", 2024-05-08T03:19:18.670 INFO:teuthology.orchestra.run.smithi177.stdout: "exporter_prio_limit": "5", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "exporter_sort_metrics": "true", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "exporter_stats_period": "5", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "fake_statfs_for_testing": "0", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "fatal_signal_handlers": "true", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "filer_max_purge_ops": "10", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "filer_max_truncate_ops": "128", 2024-05-08T03:19:18.671 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_apply_finisher_threads": "1", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_blackhole": "false", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_btrfs_clone_range": "true", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_btrfs_snap": "true", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_caller_concurrency": "10", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_collect_device_partition_information": "true", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_commit_timeout": "600.000000", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_debug_inject_read_err": "false", 2024-05-08T03:19:18.672 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_debug_omap_check": "false", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_debug_random_read_err": "0.000000", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_debug_verify_split": "false", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_dump_file": "", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fadvise": "true", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fail_eio": "true", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fd_cache_shards": "16", 2024-05-08T03:19:18.673 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fd_cache_size": "128", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fiemap": "false", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fiemap_threshold": "4096", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_index_retry_probability": "0.000000", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_inject_stall": "0", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_journal_parallel": "false", 2024-05-08T03:19:18.674 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_journal_trailing": "false", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_journal_writeahead": "false", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_kill_at": "0", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattr_size": "0", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-05-08T03:19:18.675 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-05-08T03:19:18.676 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattrs": "0", 2024-05-08T03:19:18.676 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-05-08T03:19:18.676 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattrs_other": "2", 2024-05-08T03:19:18.676 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-05-08T03:19:18.676 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_sync_interval": "5.000000", 2024-05-08T03:19:18.676 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_xattr_value_size": "0", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_merge_threshold": "-10", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_min_sync_interval": "0.010000", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_odsync_write": "false", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_omap_backend": "rocksdb", 2024-05-08T03:19:18.677 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_omap_backend_path": "", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_omap_header_cache_size": "1024", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_ondisk_finisher_threads": "1", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_op_thread_timeout": "60", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_op_threads": "2", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_punch_hole": "false", 2024-05-08T03:19:18.678 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_max_bytes": "104857600", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-05-08T03:19:18.679 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_queue_max_ops": "50", 2024-05-08T03:19:18.680 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-05-08T03:19:18.680 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_seek_data_hole": "false", 2024-05-08T03:19:18.680 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_sloppy_crc": "false", 2024-05-08T03:19:18.680 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-05-08T03:19:18.680 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_splice": "false", 2024-05-08T03:19:18.680 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_split_multiple": "2", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_split_rand_factor": "20", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_update_to": "1000", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-05-08T03:19:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_enable": "true", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-05-08T03:19:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_xfs_extsize": "false", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "filestore_zfs_snap": "false", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "fio_dir": "/tmp/fio", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_allow_other": "true", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_atomic_o_trunc": "true", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_big_writes": "true", 2024-05-08T03:19:18.683 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_debug": "false", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_default_permissions": "false", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_disable_pagecache": "false", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_max_write": "0", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_multithreaded": "true", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_require_active_mds": "true", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_set_user_groups": "true", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_splice_move": "true", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_splice_read": "true", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_splice_write": "true", 2024-05-08T03:19:18.684 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_syncfs_on_mksnap": "true", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "fuse_use_invalidate_cb": "true", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "gss_target_name": "ceph", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_file": "", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_inject_failure": "0", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_interval": "5", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "host": "smithi177", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_max_size": "1073741824", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-05-08T03:19:18.685 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "immutable_object_cache_watermark": "0.900000", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "inject_early_sigterm": "false", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "jaeger_agent_port": "6799", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "jaeger_tracing_enable": "false", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_aio": "true", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_align_min_size": "65536", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_block_align": "true", 2024-05-08T03:19:18.686 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_block_size": "4096", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_dio": "true", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_discard": "false", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_force_aio": "false", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_ignore_corruption": "false", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_max_write_bytes": "10485760", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_max_write_entries": "100", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_replay_from": "0", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_throttle_high_multiple": "0.000000", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-05-08T03:19:18.687 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_throttle_max_multiple": "0.000000", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_write_header_frequency": "0", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "journal_zero_on_create": "false", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "journaler_prefetch_periods": "10", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "journaler_prezero_periods": "5", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "journaler_write_head_interval": "15", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "key": "", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "keyfile": "", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_backend": "rocksdb", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_default_stripe_size": "65536", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_fsck_on_mount": "false", 2024-05-08T03:19:18.688 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_fsck_on_mount_deep": "true", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_max_bytes": "67108864", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_max_ops": "512", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_nid_prealloc": "1024", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_onode_map_size": "1024", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_sync_submit_transaction": "false", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "kstore_sync_transaction": "false", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "librados_thread_count": "2", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "lockdep": "false", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "lockdep_force_backtrace": "false", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "log_coarse_timestamps": "true", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "log_flush_on_exit": "false", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "log_graylog_host": "127.0.0.1", 2024-05-08T03:19:18.689 INFO:teuthology.orchestra.run.smithi177.stdout: "log_graylog_port": "12201", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_max_new": "1000", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_max_recent": "10000", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_stderr_prefix": "", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_stop_at_utilization": "0.970000", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_to_file": "true", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_to_graylog": "false", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_to_journald": "false", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_to_stderr": "false", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "log_to_syslog": "false", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "max_rotating_auth_attempts": "10", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_action_on_write_error": "1", 2024-05-08T03:19:18.690 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_alternate_name_max": "8192", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_asio_thread_count": "2", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_export_pin": "true", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_fragment_dirs": "true", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_fragment_interval": "5", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_fragment_size_max": "100000", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_idle_threshold": "0.000000", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_interval": "10", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_max": "-1", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_max_until": "-1", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_merge_size": "50", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_midchunk": "0.300000", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_min_rebalance": "0.100000", 2024-05-08T03:19:18.691 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_min_start": "0.200000", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_minchunk": "0.001000", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_mode": "0", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_need_max": "1.200000", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_need_min": "0.800000", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_overload_epochs": "2", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_sample_interval": "3.000000", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_split_bits": "3", 2024-05-08T03:19:18.692 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_split_rd": "25000.000000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_split_size": "10000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_split_wr": "10000.000000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_target_decay": "10.000000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_beacon_grace": "15.000000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_beacon_interval": "4.000000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_beacon_mon_down_grace": "60", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_memory_limit": "4294967296", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_mid": "0.700000", 2024-05-08T03:19:18.693 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_quiesce_delay": "0", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_quiesce_sleep": "200", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_quiesce_splitauth": "true", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_quiesce_threshold": "524288", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_release_free_interval": "10", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_reservation": "0.050000", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_trim_interval": "1", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache_trim_threshold": "262144", 2024-05-08T03:19:18.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_client_delegate_inos_pct": "50", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_client_prealloc_inos": "1000", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_connect_bootstrapping": "false", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_damage_table_max_entries": "10000", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_debug_auth_pins": "false", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_debug_frag": "false", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_debug_scatterstat": "false", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_debug_subtrees": "false", 2024-05-08T03:19:18.695 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_decay_halflife": "5.000000", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_default_dir_hash": "2", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_defer_session_stale": "true", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_deny_all_reconnect": "false", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dir_keys_per_op": "16384", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dir_max_commit_size": "10", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dir_max_entries": "0", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dir_prefetch": "true", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dirstat_min_interval": "1.000000", 2024-05-08T03:19:18.696 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dump_cache_after_rejoin": "false", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dump_cache_on_map": "false", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dump_cache_threshold_file": "0", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_early_reply": "true", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_enable_op_tracker": "true", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_enforce_unique_name": "true", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_export_ephemeral_distributed": "true", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_export_ephemeral_random": "true", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_forward_all_requests_to_auth": "false", 2024-05-08T03:19:18.697 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_health_cache_threshold": "1.500000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_health_summarize_threshold": "10", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_heartbeat_grace": "15.000000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_heartbeat_reset_grace": "1000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_inject_health_dummy": "false", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_inject_migrator_session_race": "false", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-05-08T03:19:18.698 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_join_fs": "", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_journal_format": "1", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_create_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_export_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_import_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_journal_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_journal_expire_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_journal_replay_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_link_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_mdstable_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_openc_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_rename_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_shutdown_at": "0", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-05-08T03:19:18.699 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_event_large_threshold": "524288", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_events_per_segment": "1024", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_major_segment_event_ratio": "12", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_max_events": "-1", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_max_segments": "128", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_pause": "false", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_segment_size": "0", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_skip_corrupt_events": "false", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_skip_unbounded_events": "false", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_trim_threshold": "128", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log_warn_factor": "2.000000", 2024-05-08T03:19:18.700 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_caps_per_client": "1048576", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_completed_flushes": "100000", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_completed_requests": "100000", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_export_size": "20971520", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_file_recover": "32", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_purge_files": "64", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_purge_ops": "8192", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_max_snaps_per_dir": "100", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_metrics_update_interval": "2", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_min_caps_per_client": "100", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_min_caps_working_set": "10000", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_numa_node": "-1", 2024-05-08T03:19:18.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_op_complaint_time": "30.000000", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_op_history_duration": "600", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_op_history_size": "20", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_op_history_slow_op_size": "20", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_op_history_slow_op_threshold": "10", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_op_log_threshold": "5", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_ping_grace": "15", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_ping_interval": "5", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-05-08T03:19:18.702 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_recall_max_caps": "30000", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_recall_max_decay_threshold": "131072", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_recall_warning_threshold": "262144", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_reconnect_timeout": "45.000000", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_replay_interval": "1.000000", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-05-08T03:19:18.703 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_root_ino_gid": "0", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_root_ino_uid": "0", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_blocklist_on_evict": "true", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_blocklist_on_timeout": "true", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_session_metadata_threshold": "16777216", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_shutdown_check": "0", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_skip_ino": "0", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sleep_rank_change": "0.000000", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_snap_max_uid": "4294967294", 2024-05-08T03:19:18.704 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_snap_min_uid": "0", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_snap_rstat": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_standby_replay_damaged": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_symlink_recovery": "true", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_task_status_update_interval": "2.000000", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_thrash_exports": "0", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_thrash_fragments": "0", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_tick_interval": "5.000000", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_valgrind_exit": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_verify_backtrace": "1", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_verify_scatter": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_wipe_ino_prealloc": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_wipe_sessions": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool_debug": "false", 2024-05-08T03:19:18.705 INFO:teuthology.orchestra.run.smithi177.stdout: "memstore_debug_omit_block_device_write": "false", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "memstore_device_bytes": "1073741824", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "memstore_page_set": "false", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "memstore_page_size": "65536", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_client_bytes": "134217728", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_client_messages": "512", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_connect_retry_interval": "1.000000", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_disabled_modules": "", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_max_pg_num_change": "128", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_mds_bytes": "134217728", 2024-05-08T03:19:18.706 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_mds_messages": "128", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_mon_bytes": "134217728", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_mon_messages": "128", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_osd_bytes": "536870912", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_osd_messages": "8192", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_pool": "true", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_service_beacon_grace": "60.000000", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_standby_modules": "true", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_stats_period": "5", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_stats_threshold": "5", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_tick_period": "2", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_accept_timeout_factor": "2.000000", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_allow_pool_delete": "true", 2024-05-08T03:19:18.707 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_allow_pool_size_one": "true", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_auth_validate_all_caps": "true", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_bytes": "104857600", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_directed_command_retry": "5", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_hunt_interval": "3.000000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_hunt_parallel": "3", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_log_interval": "1.000000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_ping_interval": "10.000000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_ping_timeout": "30.000000", 2024-05-08T03:19:18.708 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_client_target_rank": "-1", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_clock_drift_allowed": "1.000000", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_file_level": "debug", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_file": "true", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_graylog": "false", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_journald": "false", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_stderr": "false", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_compact_on_bootstrap": "false", 2024-05-08T03:19:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_compact_on_start": "false", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_compact_on_trim": "true", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_con_tracker_persist_interval": "10", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_con_tracker_score_halflife": "43200", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_config_key_max_entry_size": "65536", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_cpu_threads": "4", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_crush_min_required_version": "hammer", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_daemon_bytes": "419430400", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_data_avail_crit": "5", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_data_avail_warn": "5", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_data_size_warn": "16106127360", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_block_osdmap_trim": "false", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-05-08T03:19:18.710 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_dump_json": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_dump_transactions": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_extra_checks": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_no_require_reef": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_no_require_squid": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_delta_reset_interval": "10.000000", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_dns_srv_name": "ceph-mon", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_down_mkfs_grace": "120", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_down_uptime_grace": "60", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_election_default_strategy": "1", 2024-05-08T03:19:18.711 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_election_timeout": "5.000000", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_elector_ping_divisor": "2", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_elector_ping_timeout": "2.000000", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_enable_op_tracker": "true", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_fake_pool_delete": "false", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_force_quorum_join": "false", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_globalid_prealloc": "10000", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_health_detail_to_clog": "false", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_health_log_update_period": "5", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_health_max_detail": "50", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_health_to_clog": "true", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_health_to_clog_interval": "600", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_host": "172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790],172.21.15.188", 2024-05-08T03:19:18.712 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_host_override": "", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_initial_members": "", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_keyvaluedb": "rocksdb", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_lease": "5.000000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_log_full_interval": "50", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_log_max": "10000", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_log_max_summary": "50", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_log_entries_per_event": "4096", 2024-05-08T03:19:18.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_log_epochs": "500", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_mdsmap_epochs": "500", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_mgrmap_epochs": "500", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_osd": "10000", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_pg_per_osd": "10000", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_pool_pg_num": "65536", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mds_force_trim_to": "0", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mds_skip_sanity": "false", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_memory_autotune": "true", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_memory_target": "2147483648", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mgr_beacon_grace": "30", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mgr_digest_period": "5", 2024-05-08T03:19:18.714 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mgr_inactive_grace": "60", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mgr_mkfs_grace": "240", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_min_osdmap_epochs": "500", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_op_complaint_time": "30", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_op_history_duration": "600", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_op_history_size": "20", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_op_history_slow_op_size": "20", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_op_history_slow_op_threshold": "10", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_op_log_threshold": "5", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_auto_mark_in": "false", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_auto_mark_new_in": "true", 2024-05-08T03:19:18.715 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_cache_size": "500", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_cache_size_min": "134217728", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_crush_smoke_test": "true", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_destroyed_out_interval": "600", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_down_out_interval": "600", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_force_trim_to": "0", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_full_ratio": "0.950000", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_laggy_halflife": "3600", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_laggy_max_interval": "300", 2024-05-08T03:19:18.716 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_laggy_weight": "0.300000", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_max_creating_pgs": "1024", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_max_initial_pgs": "1024", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_min_down_reporters": "2", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_prime_pg_temp": "true", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_report_timeout": "900", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_warn_num_repaired": "10", 2024-05-08T03:19:18.717 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osd_warn_op_age": "32.000000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osdmap_full_prune_interval": "10", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osdmap_full_prune_min": "10000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pg_stuck_threshold": "60", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pg_warn_min_objects": "10000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pg_warn_min_per_osd": "0", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pool_quota_crit_threshold": "0", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_pool_quota_warn_threshold": "0", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_probe_timeout": "2.000000", 2024-05-08T03:19:18.718 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_reweight_max_change": "0.050000", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_reweight_max_osds": "4", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_scrub_interval": "86400", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_scrub_max_keys": "100", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_scrub_timeout": "300", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_session_timeout": "300", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_smart_report_timeout": "5", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_stat_smooth_intervals": "6", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_stretch_pool_min_size": "2", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_stretch_pool_size": "4", 2024-05-08T03:19:18.719 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_subscribe_interval": "86400.000000", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_sync_debug": "false", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_sync_max_payload_keys": "2000", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_sync_max_payload_size": "1048576", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_sync_provider_kill_at": "0", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_sync_requester_kill_at": "0", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_sync_timeout": "60.000000", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_target_pg_per_osd": "100", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_tick_interval": "5", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_timecheck_interval": "300.000000", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_use_min_delay_socket": "false", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_older_version_delay": "604800", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-05-08T03:19:18.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_filestore_osds": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_misplaced": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_older_version": "true", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_pool_no_app": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-05-08T03:19:18.721 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_on_too_few_osds": "false", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "monmap": "", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "motr_tracing_enabled": "false", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_op_threads": "3", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_buffer_size": "131072", 2024-05-08T03:19:18.722 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_cm": "false", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_device_name": "", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_dscp": "96", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_gid_idx": "0", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_local_gid": "", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_polling_us": "1000", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_port_num": "1", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_roce_ver": "1", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_send_buffers": "1024", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_sl": "3", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_support_srq": "true", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_rdma_type": "ib", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_async_reap_threshold": "5", 2024-05-08T03:19:18.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_before_connect": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_ipv4": "true", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_ipv6": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_msgr1": "true", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_msgr2": "true", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_port_max": "7568", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_port_min": "6800", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_prefer_ipv4": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_retry_count": "3", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_bind_retry_delay": "5", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_blackhole_client": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_blackhole_mds": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_blackhole_mgr": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_blackhole_mon": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_blackhole_osd": "false", 2024-05-08T03:19:18.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_client_mode": "crc secure", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_cluster_mode": "crc secure", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_cluster_type": "", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_compress_secure": "false", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_connection_idle_timeout": "900", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_connection_ready_timeout": "10", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_crc_data": "true", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_crc_header": "true", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_die_on_bad_msg": "false", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_die_on_bug": "true", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_die_on_old_message": "true", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_die_on_skipped_message": "false", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_die_on_unhandled_msg": "false", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-05-08T03:19:18.725 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_coremask": "0xF", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_devs_allowlist": "", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_enable_tso": "true", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_hugepages": "", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_hw_flow_control": "true", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_lro": "true", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_memory_channel": "4", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_pmd": "", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_port_id": "0", 2024-05-08T03:19:18.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dump_corrupt_message_level": "1", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_dump_on_send": "false", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_initial_backoff": "0.200000", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_inject_delay_max": "1.000000", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_inject_delay_probability": "0.000000", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_inject_delay_type": "", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_inject_internal_delays": "0.000000", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_inject_network_congestion": "0", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_inject_socket_failures": "5000", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_learn_addr_from_peer": "true", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_max_accept_failures": "4", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_max_backoff": "15.000000", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_mon_client_mode": "secure crc", 2024-05-08T03:19:18.727 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_mon_cluster_mode": "secure crc", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_mon_service_mode": "secure crc", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_osd_compress_min_size": "1024", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_osd_compress_mode": "none", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_osd_compression_algorithm": "snappy", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_pq_min_cost": "65536", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_public_type": "", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_service_mode": "crc secure", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_tcp_listen_backlog": "512", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_tcp_nodelay": "true", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_tcp_rcvbuf": "0", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "ms_type": "async+posix", 2024-05-08T03:19:18.728 INFO:teuthology.orchestra.run.smithi177.stdout: "no_config_file": "false", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_completion_locks_per_session": "32", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_debug_inject_relock_delay": "false", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_inflight_op_bytes": "104857600", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_inflight_ops": "1024", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_inject_no_watch_ping": "false", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_tick_interval": "5.000000", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter_timeout": "10.000000", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objectstore_blackhole": "false", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "openssl_engine_opts": "", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_delay_time": "5.000000", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_hist_halflife": "1000", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_max_low_ops": "2", 2024-05-08T03:19:18.729 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_max_ops": "4", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_quantize_effort": "0.100000", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_agent_slop": "0.020000", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_allow_recovery_below_min_size": "true", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_asio_thread_count": "2", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_async_recovery_min_cost": "100", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_auto_mark_unfound_lost": "false", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_backfill_retry_interval": "30.000000", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_backfill_scan_max": "512", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_backfill_scan_min": "64", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_backoff_on_degraded": "false", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_backoff_on_peering": "false", 2024-05-08T03:19:18.730 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_backoff_on_unfound": "true", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_beacon_report_interval": "300", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bench_duration": "30", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bench_max_block_size": "67108864", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bench_small_size_max_iops": "100", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_blkin_trace_all": "false", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_blocked_scrub_grace_period": "120", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_check_for_log_corruption": "false", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T03:19:18.731 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_class_update_on_start": "true", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_client_message_cap": "256", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_client_message_size_cap": "524288000", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_client_op_priority": "63", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_client_watch_timeout": "30", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_command_max_records": "256", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_compact_on_start": "false", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_crush_chooseleaf_type": "0", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_crush_initial_weight": "-1.000000", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_crush_update_on_start": "true", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_crush_update_weight_set": "true", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-05-08T03:19:18.732 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_drop_ping_duration": "0", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_feed_pullee": "-1", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_misdirected_ops": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_no_acting_change": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_no_purge_strays": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_op_order": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_pg_log_writeout": "false", 2024-05-08T03:19:18.733 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_pretend_recovery_active": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_shutdown": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_trim_objects": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_verify_cached_snaps": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_verify_missing_on_start": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_verify_snaps": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_keys": "1024", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-05-08T03:19:18.734 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_stride": "524288", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_default_data_pool_replay_window": "5", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_default_notify_timeout": "30", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_delete_sleep": "0.000000", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_discard_disconnected_ops": "true", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_enable_op_tracker": "true", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_extblkdev_plugins": "vdo", 2024-05-08T03:19:18.735 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_fast_info": "true", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_fast_shutdown": "true", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_fast_shutdown_timeout": "15", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_function_tracing": "false", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_grace": "20", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_interval": "6", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_min_peers": "10", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_min_size": "2000", 2024-05-08T03:19:18.736 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_stale": "600", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_hit_set_max_size": "100000", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_hit_set_min_size": "1000", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_journal_flush_on_shutdown": "true", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_journal_size": "5120", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_kill_backfill_at": "0", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_loop_before_reset_tphandle": "64", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_map_cache_size": "50", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_map_dedup": "true", 2024-05-08T03:19:18.737 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_map_message_max": "40", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_map_message_max_bytes": "10485760", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_map_share_max_epochs": "40", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_attr_name_len": "100", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_attr_size": "0", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_backfills": "1", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_markdown_count": "5", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_markdown_period": "600", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_object_name_len": "2048", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_object_namespace_len": "256", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_object_size": "134217728", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_omap_entries_per_request": "1024", 2024-05-08T03:19:18.738 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_pg_blocked_by": "16", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_pg_log_entries": "10000", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_pgls": "1024", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_push_cost": "8388608", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_push_objects": "10", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_scrubs": "3", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_trimming_pgs": "2", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_write_op_reply_len": "64", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_max_write_size": "90", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-05-08T03:19:18.739 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_override_recovery_settings": "false", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_profile": "balanced", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mclock_skip_benchmark": "false", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_base": "805306368", 2024-05-08T03:19:18.740 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_cache_min": "134217728", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_target": "4294967296", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_target_autotune": "false", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_min_pg_log_entries": "250", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_min_recovery_priority": "0", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mon_heartbeat_interval": "30", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mon_report_interval": "5", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mon_report_max_in_flight": "2", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_num_cache_shards": "32", 2024-05-08T03:19:18.741 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_num_op_tracker_shard": "32", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_numa_auto_affinity": "true", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_numa_node": "-1", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_numa_prefer_iface": "true", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_objecter_finishers": "1", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_objectstore": "bluestore", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_objectstore_fuse": "false", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_objectstore_tracing": "false", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_complaint_time": "30.000000", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_history_duration": "600", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_history_size": "20", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_history_slow_op_size": "20", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_log_threshold": "5", 2024-05-08T03:19:18.742 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_num_shards": "0", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_num_shards_hdd": "5", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_num_shards_ssd": "8", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_num_threads_per_shard": "0", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_pq_min_cost": "65536", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_queue": "mclock_scheduler", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_queue_cut_off": "high", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_thread_suicide_timeout": "150", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_op_thread_timeout": "15", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_open_classes_on_start": "true", 2024-05-08T03:19:18.743 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_os_flags": "0", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_peering_op_priority": "255", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_delete_cost": "1048576", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_delete_priority": "5", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_log_dups_tracked": "3000", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_log_trim_max": "10000", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_log_trim_min": "100", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_object_context_cache_count": "64", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-05-08T03:19:18.744 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_crimson": "false", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_crush_rule": "-1", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_ec_fast_read": "false", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_flag_bulk": "false", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_flag_nodelete": "false", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_flags": "0", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_min_size": "0", 2024-05-08T03:19:18.745 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_pg_num": "32", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_pgp_num": "0", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_read_ratio": "70", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_size": "2", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_default_type": "replicated", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pool_use_gmt_hitset": "true", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_push_per_object_cost": "1000", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_read_ec_check_for_errors": "false", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recover_clone_overlap": "true", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recover_clone_overlap_limit": "10", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_cost": "20971520", 2024-05-08T03:19:18.746 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_delay_start": "0.000000", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_max_active": "0", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_max_active_hdd": "3", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_max_active_ssd": "10", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_max_chunk": "8388608", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_max_single_start": "1", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_op_priority": "3", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_op_warn_multiple": "16", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_priority": "5", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_retry_interval": "30.000000", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_sleep": "0.000000", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-05-08T03:19:18.747 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_repair_during_recovery": "false", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_requested_scrub_priority": "120", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_rollback_to_cluster_snap": "", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_auto_repair": "false", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_begin_hour": "0", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_begin_week_day": "0", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_chunk_max": "25", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_chunk_min": "5", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_cost": "52428800", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_during_recovery": "false", 2024-05-08T03:19:18.748 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_end_hour": "0", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_end_week_day": "0", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_event_cost": "4096", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_invalid_stats": "true", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_load_threshold": "0.500000", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_max_interval": "604800.000000", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_max_preemptions": "5", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_min_interval": "86400.000000", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_priority": "5", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_scrub_sleep": "0.000000", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_shutdown_pgref_assert": "false", 2024-05-08T03:19:18.749 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_skip_data_digest": "false", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_smart_report_timeout": "5", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_snap_trim_cost": "1048576", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_snap_trim_priority": "5", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_snap_trim_sleep": "0.000000", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_stats_update_period_scrubbing": "15", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_target_transaction_size": "30", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-05-08T03:19:18.750 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_default_cache_mode": "writeback", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_tracing": "false", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_use_stale_snap": "false", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "osdc_blkin_trace_all": "false", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_kill_at": "0", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_max_join_drift": "10", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_min": "500", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_min_wait": "0.050000", 2024-05-08T03:19:18.751 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_propose_interval": "1.000000", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_service_trim_max": "500", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_service_trim_max_multiplier": "20", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_service_trim_min": "250", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_stash_full_interval": "25", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_trim_max": "500", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "paxos_trim_min": "250", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "perf": "true", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "-", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrv": "", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "public_bind_addr": "-", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "public_network": "", 2024-05-08T03:19:18.752 INFO:teuthology.orchestra.run.smithi177.stdout: "public_network_interface": "", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "qat_compressor_enabled": "false", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "qat_compressor_session_max_number": "256", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rados_mon_op_timeout": "0", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rados_osd_op_timeout": "0", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rados_tracing": "false", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_atime_update_interval": "60", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_balance_parent_reads": "false", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_balance_snap_reads": "false", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_blkin_trace_all": "false", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_blocklist_expire_seconds": "0", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_blocklist_on_break_lock": "true", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache": "true", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_block_writes_upfront": "false", 2024-05-08T03:19:18.753 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_max_dirty": "25165824", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_max_dirty_object": "0", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_policy": "writearound", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_size": "33554432", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_target_dirty": "16777216", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_clone_copy_on_read": "false", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_compression_hint": "none", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_concurrent_management_ops": "10", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_clone_format": "auto", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_data_pool": "", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_features": "61", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_format": "2", 2024-05-08T03:19:18.754 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_map_options": "", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_order": "22", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_pool": "rbd", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_stripe_count": "0", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_default_stripe_unit": "0", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_disable_zero_copy_writes": "true", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_discard_granularity_bytes": "65536", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_enable_alloc_hint": "true", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_io_scheduler": "simple", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_commit_age": "5.000000", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-05-08T03:19:18.755 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_object_flush_interval": "0", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_order": "24", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_pool": "", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_journal_splay_width": "4", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_localize_parent_reads": "false", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_localize_snap_reads": "false", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-05-08T03:19:18.756 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_image_policy_type": "simple", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_autotune": "true", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_base": "805306368", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-05-08T03:19:18.757 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_memory_target": "4294967296", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirroring_delete_delay": "0", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirroring_replay_delay": "0", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_move_to_trash_on_remove": "false", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_mtime_update_interval": "60", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_non_blocking_aio": "true", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_op_thread_timeout": "60", 2024-05-08T03:19:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_op_threads": "1", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_parent_cache_enabled": "false", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_persistent_cache_mode": "disabled", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_persistent_cache_path": "/tmp", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_persistent_cache_size": "1073741824", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_plugins": "", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_bps_burst": "0", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_bps_limit": "0", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_exclude_ops": "0", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_iops_burst": "0", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_iops_limit": "0", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_read_bps_burst": "0", 2024-05-08T03:19:18.759 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_read_bps_limit": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_read_iops_burst": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_read_iops_limit": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_schedule_tick_min": "50", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_write_bps_burst": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_write_bps_limit": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_write_iops_burst": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_qos_write_iops_limit": "0", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_quiesce_notification_attempts": "10", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_read_from_replica_policy": "default", 2024-05-08T03:19:18.760 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_readahead_max_bytes": "524288", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_readahead_trigger_requests": "10", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_request_timed_out_seconds": "30", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_skip_partial_discard": "true", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_tracing": "false", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_validate_names": "true", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd_validate_pool": "true", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "restapi_base_url": "", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "restapi_log_level": "", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_account_default_quota_max_size": "-1", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_acl_grants_max_num": "100", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_admin_entry": "admin", 2024-05-08T03:19:18.761 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_backend_store": "rados", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_barbican_url": "", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_beast_enable_async": "true", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_counters_cache": "false", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_index_max_aio": "128", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_quota_ttl": "600", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_cache_enabled": "true", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_cache_expiry_interval": "900", 2024-05-08T03:19:18.762 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_cache_lru_size": "25000", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_config_store": "rados", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_content_length_compat": "false", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_copy_obj_progress": "true", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_cors_rules_max_num": "100", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_cross_domain_policy": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_default_encryption_key": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_addr": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_ca_path": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_client_cert": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_client_key": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_password": "", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-05-08T03:19:18.763 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_kmip_username": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_require_ssl": "true", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-05-08T03:19:18.764 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_suppress_logs": "true", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_addr": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_auth": "token", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_namespace": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_prefix": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_token_file": "", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_curl_buffersize": "524288", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_curl_low_speed_limit": "1024", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_curl_low_speed_time": "300", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_curl_tcp_keepalive": "0", 2024-05-08T03:19:18.765 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_l1_fadvise": "4", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d4n_host": "127.0.0.1", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_d4n_port": "6379", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_log_changes_size": "1000", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_log_num_shards": "128", 2024-05-08T03:19:18.766 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_log_window": "30", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_notify_interval_msec": "0", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_sync_poll_interval": "20", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_data_sync_spawn_window": "20", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_default_data_log_backing": "fifo", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_default_region_info_oid": "default.region", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_defer_to_bucket_acls": "", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-05-08T03:19:18.767 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_data_res": "500.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dns_name": "", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dns_s3website_name": "", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_dynamic_resharding": "true", 2024-05-08T03:19:18.768 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_gc_threads": "true", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_lc_threads": "true", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_ops_log": "false", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_quota_threads": "true", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_static_website": "false", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enable_usage_log": "false", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_enforce_swift_acls": "true", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_exit_timeout_secs": "120", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_expose_bucket": "false", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_extended_http_attrs": "", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_filter": "none", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_frontends": "beast port=7480", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_max_concurrent_io": "10", 2024-05-08T03:19:18.769 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_max_deferred": "50", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_max_objs": "32", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_max_queue_size": "134213632", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_max_trim_chunk": "16", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_obj_min_wait": "7200", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_processor_max_time": "3600", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_gc_processor_period": "3600", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_get_obj_window_size": "16777216", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_healthcheck_disabling_path": "", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ignore_get_invalid_range": "false", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_init_timeout": "300", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-05-08T03:19:18.770 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_kafka_connection_idle": "300", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_kafka_message_timeout": "5000", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_kafka_sleep_timeout": "10", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_domain": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_password": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_password_path": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_project": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_tenant": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_token": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_token_path": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_admin_user": "", 2024-05-08T03:19:18.771 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_api_version": "2", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_barbican_domain": "", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_barbican_password": "", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_barbican_project": "", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_barbican_tenant": "", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_barbican_user": "", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_implicit_tenants": "false", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_service_token_enabled": "false", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_token_cache_size": "10000", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_url": "", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_keystone_verify_ssl": "true", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_debug_interval": "-1", 2024-05-08T03:19:18.772 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_lock_max_time": "90", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_max_objs": "32", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_max_rules": "1000", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_max_worker": "3", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_max_wp_worker": "3", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lc_thread_delay": "0", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ldap_dnattr": "uid", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-05-08T03:19:18.773 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ldap_searchfilter": "", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ldap_uri": "", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_log_http_headers": "", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_log_nonexistent_bucket": "false", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_log_object_name_utc": "false", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_attr_name_len": "0", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_attr_size": "0", 2024-05-08T03:19:18.774 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_attrs_num_in_req": "0", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_chunk_size": "4194304", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_concurrent_requests": "1024", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_dynamic_shards": "1999", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_listing_results": "1000", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_notify_retries": "10", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_objs_per_shard": "100000", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_put_param_size": "1048576", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_put_size": "5368709120", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_max_slo_entries": "1000", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_md_log_max_shards": "64", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_md_notify_interval_msec": "200", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_meta_sync_poll_interval": "20", 2024-05-08T03:19:18.775 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_meta_sync_spawn_window": "20", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_mp_lock_max_time": "600", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_multipart_min_part_size": "5242880", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_fhcache_size": "2017", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_lru_lanes": "5", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_max_gc": "300", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_run_gc_threads": "false", 2024-05-08T03:19:18.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_run_lc_threads": "false", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_run_quota_threads": "false", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_run_sync_thread": "false", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_num_async_rados_threads": "32", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_num_control_oids": "8", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_numa_node": "-1", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_obj_stripe_size": "4194304", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_objexp_chunk_size": "100", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_objexp_gc_interval": "600", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_objexp_hints_num_shards": "127", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-05-08T03:19:18.777 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_op_thread_timeout": "600", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_op_tracing": "false", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_opa_token": "", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_opa_url": "", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_opa_verify_ssl": "true", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ops_log_rados": "false", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_ops_log_socket_path": "", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_period_push_interval": "2.000000", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_period_push_interval_max": "30.000000", 2024-05-08T03:19:18.778 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_period_root_pool": ".rgw.root", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_posix_cache_lanes": "3", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_posix_cache_max_buckets": "100", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_posix_cache_partitions": "3", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_print_continue": "true", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_print_prohibited_content_length": "false", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_rados_tracing": "false", 2024-05-08T03:19:18.779 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_realm": "", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_realm_id": "", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_region": "", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_region_root_pool": ".rgw.root", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_relaxed_region_enforcement": "false", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_request_uri": "", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_reshard_batch_size": "64", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_reshard_max_aio": "128", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_reshard_num_logs": "16", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_reshard_thread_interval": "600", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_resolve_cname": "false", 2024-05-08T03:19:18.780 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_rest_getusage_op_compat": "false", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_run_sync_thread": "true", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_auth_use_keystone": "false", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_auth_use_ldap": "false", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_auth_use_rados": "true", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_auth_use_sts": "false", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_s3_success_create_obj_status": "0", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_scheduler_type": "throttler", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_script_uri": "", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_service_provider_name": "", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_client_id": "", 2024-05-08T03:19:18.781 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_client_secret": "", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_entry": "sts", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_key": "sts", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_max_session_duration": "43200", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_min_session_duration": "900", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sts_token_introspection_url": "", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_account_in_url": "false", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_auth_entry": "auth", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_auth_url": "", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_custom_header": "", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_enforce_content_length": "false", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_need_stats": "true", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_tenant_name": "", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_token_expiration": "86400", 2024-05-08T03:19:18.782 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_url": "", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_url_prefix": "swift", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_swift_versioning_enabled": "false", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_lease_period": "120", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_log_trim_interval": "1200", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_obj_etag_verify": "false", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_trace_history_size": "4096", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-05-08T03:19:18.783 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_thread_pool_size": "512", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_topic_persistency_max_retries": "0", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_topic_require_publish_policy": "false", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_comment": "", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_createby": "", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_encoding": "", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_flag": "false", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_max_size": "5368709120", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_origin": "", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_sha_unit": "524288", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_torrent_tracker": "", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_trust_forwarded_https": "false", 2024-05-08T03:19:18.784 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_usage_log_tick_interval": "30", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_usage_max_shards": "32", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_usage_max_user_shards": "1", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_use_opa_authz": "false", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_counters_cache": "false", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_counters_cache_size": "10000", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_default_quota_max_size": "-1", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_max_buckets": "1000", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_policies_max_num": "100", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_quota_sync_interval": "86400", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-05-08T03:19:18.785 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_user_unique_email": "true", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_verify_ssl": "true", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_website_routing_rules_max_num": "50", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_zone": "", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_zone_id": "", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_zonegroup": "", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_zonegroup_id": "", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgwlc_auto_session_clear": "true", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rgwlc_skip_bucket_step": "false", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_block_size": "4096", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-05-08T03:19:18.786 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cache_shard_bits": "4", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cache_size": "536870912", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cache_type": "binned_lru", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_collect_compaction_stats": "false", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_collect_extended_stats": "false", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_collect_memory_stats": "false", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_index_type": "binary_search", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_log_to_ceph_log": "true", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_metadata_block_size": "4096", 2024-05-08T03:19:18.787 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_partition_filters": "false", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_perf": "false", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "rotating_keys_renewal_timeout": "10", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "run_dir": "/var/run/ceph", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_block_create": "true", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_cache_lru_size": "67108864", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_cbjournal_size": "5368709120", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_data_delta_based_overwrite": "0", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_default_max_object_size": "16777216", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_device_size": "53687091200", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_journal_batch_capacity": "16", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-05-08T03:19:18.788 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_journal_iodepth_limit": "5", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_main_device_type": "SSD", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_max_concurrent_transactions": "8", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "seastore_segment_size": "67108864", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "set_keepcaps": "false", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "setgroup": "", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "setuser": "", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "setuser_match_path": "", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_misplaced_ratio": "0.050000", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "thp": "false", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "threadpool_default_timeout": "60", 2024-05-08T03:19:18.789 INFO:teuthology.orchestra.run.smithi177.stdout: "threadpool_empty_queue_max_wait": "2", 2024-05-08T03:19:18.790 INFO:teuthology.orchestra.run.smithi177.stdout: "throttler_perf_counter": "true" 2024-05-08T03:19:18.790 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:18.792 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-05-08T03:19:18.906 INFO:teuthology.orchestra.run.smithi177.stdout:mon.a: {} 2024-05-08T03:19:18.906 INFO:teuthology.orchestra.run.smithi177.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T03:19:18.916 INFO:teuthology.orchestra.run.smithi177.stdout:mon.b: {} 2024-05-08T03:19:18.916 INFO:teuthology.orchestra.run.smithi177.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T03:19:18.924 INFO:teuthology.orchestra.run.smithi177.stdout:mon.c: {} 2024-05-08T03:19:18.925 INFO:teuthology.orchestra.run.smithi177.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T03:19:18.935 DEBUG:teuthology.orchestra.run.smithi177:> type ceph-objectstore-tool 2024-05-08T03:19:18.950 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T03:19:18.950 DEBUG:teuthology.orchestra.run.smithi188:> type ceph-objectstore-tool 2024-05-08T03:19:18.969 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T03:19:18.969 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-05-08T03:19:18.980 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-05-08T03:19:18.980 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-05-08T03:19:18.980 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:18.980 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:18.980 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:19:18.981 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-05-08T03:19:18.981 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-05-08T03:19:18.981 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:18.982 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-05-08T03:19:18.982 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:18.982 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-05-08T03:19:18.982 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:19:18.983 DEBUG:teuthology.orchestra.run.smithi177:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-05-08T03:19:18.992 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:18.990+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:19.032 INFO:teuthology.orchestra.run.smithi177.stdout:mapped writes DISABLED 2024-05-08T03:19:19.032 INFO:teuthology.orchestra.run.smithi177.stdout:mapped reads DISABLED 2024-05-08T03:19:19.032 INFO:teuthology.orchestra.run.smithi177.stdout:Seed set to 7348 2024-05-08T03:19:19.083 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:19.081+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:19.128 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:19.128 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:19.130 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.130 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.140 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:19.148 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:19.149 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:19.168 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:19.168 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:19.185 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:19.183+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:19.202 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:19.203 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:19.238 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:19.238 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:19.249 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.249 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.258 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:19.275 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:19.275 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:19.285 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:19.284+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:19.310 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:19.310 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:19.347 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:19.347 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:19.383 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.383 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.386 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:19.384+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:19.392 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:19.400 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:19.401 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:19:19.413 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-08T03:19:18.174803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219356,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755364,"statfs":{"total":773094113280,"available":772869492736,"internally_reserved":0,"allocated":2535424,"data_stored":1601068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":60,"num_write_kb":1306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000669"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648309+0000","last_change":"2024-05-08T03:19:12.731926+0000","last_active":"2024-05-08T03:19:14.648309+0000","last_peered":"2024-05-08T03:19:14.648309+0000","last_clean":"2024-05-08T03:19:14.648309+0000","last_became_active":"2024-05-08T03:19:12.731504+0000","last_became_peered":"2024-05-08T03:19:12.731504+0000","last_unstale":"2024-05-08T03:19:14.648309+0000","last_undegraded":"2024-05-08T03:19:14.648309+0000","last_fullsized":"2024-05-08T03:19:14.648309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:06:07.983173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648323+0000","last_change":"2024-05-08T03:19:12.966376+0000","last_active":"2024-05-08T03:19:14.648323+0000","last_peered":"2024-05-08T03:19:14.648323+0000","last_clean":"2024-05-08T03:19:14.648323+0000","last_became_active":"2024-05-08T03:19:12.966006+0000","last_became_peered":"2024-05-08T03:19:12.966006+0000","last_unstale":"2024-05-08T03:19:14.648323+0000","last_undegraded":"2024-05-08T03:19:14.648323+0000","last_fullsized":"2024-05-08T03:19:14.648323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:19.151925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648911+0000","last_change":"2024-05-08T03:19:12.731258+0000","last_active":"2024-05-08T03:19:14.648911+0000","last_peered":"2024-05-08T03:19:14.648911+0000","last_clean":"2024-05-08T03:19:14.648911+0000","last_became_active":"2024-05-08T03:19:12.730838+0000","last_became_peered":"2024-05-08T03:19:12.730838+0000","last_unstale":"2024-05-08T03:19:14.648911+0000","last_undegraded":"2024-05-08T03:19:14.648911+0000","last_fullsized":"2024-05-08T03:19:14.648911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:09.285333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:13.730315+0000","last_change":"2024-05-08T03:19:12.732433+0000","last_active":"2024-05-08T03:19:13.730315+0000","last_peered":"2024-05-08T03:19:13.730315+0000","last_clean":"2024-05-08T03:19:13.730315+0000","last_became_active":"2024-05-08T03:19:12.732130+0000","last_became_peered":"2024-05-08T03:19:12.732130+0000","last_unstale":"2024-05-08T03:19:13.730315+0000","last_undegraded":"2024-05-08T03:19:13.730315+0000","last_fullsized":"2024-05-08T03:19:13.730315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:37.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":162,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:15.155757+0000","last_change":"2024-05-08T03:19:13.735020+0000","last_active":"2024-05-08T03:19:15.155757+0000","last_peered":"2024-05-08T03:19:15.155757+0000","last_clean":"2024-05-08T03:19:15.155757+0000","last_became_active":"2024-05-08T03:19:13.734713+0000","last_became_peered":"2024-05-08T03:19:13.734713+0000","last_unstale":"2024-05-08T03:19:15.155757+0000","last_undegraded":"2024-05-08T03:19:15.155757+0000","last_fullsized":"2024-05-08T03:19:15.155757+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:54.753213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.737014+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:19:14.737014+0000","last_peered":"2024-05-08T03:19:14.737014+0000","last_clean":"2024-05-08T03:19:14.737014+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:19:14.737014+0000","last_undegraded":"2024-05-08T03:19:14.737014+0000","last_fullsized":"2024-05-08T03:19:14.737014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.741329+0000","last_change":"2024-05-08T03:19:12.966511+0000","last_active":"2024-05-08T03:19:14.741329+0000","last_peered":"2024-05-08T03:19:14.741329+0000","last_clean":"2024-05-08T03:19:14.741329+0000","last_became_active":"2024-05-08T03:19:12.965999+0000","last_became_peered":"2024-05-08T03:19:12.965999+0000","last_unstale":"2024-05-08T03:19:14.741329+0000","last_undegraded":"2024-05-08T03:19:14.741329+0000","last_fullsized":"2024-05-08T03:19:14.741329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:11.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.648095+0000","last_change":"2024-05-08T03:19:12.965962+0000","last_active":"2024-05-08T03:19:14.648095+0000","last_peered":"2024-05-08T03:19:14.648095+0000","last_clean":"2024-05-08T03:19:14.648095+0000","last_became_active":"2024-05-08T03:19:12.965629+0000","last_became_peered":"2024-05-08T03:19:12.965629+0000","last_unstale":"2024-05-08T03:19:14.648095+0000","last_undegraded":"2024-05-08T03:19:14.648095+0000","last_fullsized":"2024-05-08T03:19:14.648095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:50.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T03:19:14.741313+0000","last_change":"2024-05-08T03:19:12.966516+0000","last_active":"2024-05-08T03:19:14.741313+0000","last_peered":"2024-05-08T03:19:14.741313+0000","last_clean":"2024-05-08T03:19:14.741313+0000","last_became_active":"2024-05-08T03:19:12.966041+0000","last_became_peered":"2024-05-08T03:19:12.966041+0000","last_unstale":"2024-05-08T03:19:14.741313+0000","last_undegraded":"2024-05-08T03:19:14.741313+0000","last_fullsized":"2024-05-08T03:19:14.741313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:08.473548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":770048,"data_stored":646522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":56154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":770048,"data_stored":646522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":50374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:19:19.487 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:19.485+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:19.506 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.506 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:19.588 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:19.586+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:19.618 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.619 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.627 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:19.711 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:19.710+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:19.762 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:19:19.780 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.780 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.790 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:19.793 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:19.788+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:19.891 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:19.889+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:19.942 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:19.991 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:19.990+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:20.048 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.048 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.057 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:20.092 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:20.090+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:20.162 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.162 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.170 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:20.213 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:20.212+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:20.276 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.276 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.285 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:20.326 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:20.324+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:20.394 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:20.392+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:20.430 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.430 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.439 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:20.494 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:20.492+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:20.585 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.585 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.593 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:20.595 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:20.593+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:20.695 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:20.693+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:20.711 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.711 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.720 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:20.805 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:20.803+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:20.873 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:20.873 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:20.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:20.923 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:20.921+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:20.999 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:20.997+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:21.028 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:21.028 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:21.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:21.121 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:21.119+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:21.183 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:21.184 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:21.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:21.234 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:21.232+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:21.302 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:21.300+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:21.338 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:21.338 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:21.348 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:21.363 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:21.403 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:21.401+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:21.453 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:21.453 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:21.462 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:21.487 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:21.487 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:21.503 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:21.501+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:21.505 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:21.505 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:21.520 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:21.521 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:21.537 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:21.538 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:21.555 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:21.556 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:21.572 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:21.573 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:21.589 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:21.589 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:21.603 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:21.602+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:21.605 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:21.606 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:21.609 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:21.609 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:21.618 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:21.705 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:21.703+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:21.765 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:21.765 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:21.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:21.780 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:19:21.824 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:21.823+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:21.906 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:21.904+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:21.921 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:21.922 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:21.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:21.922 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:19:21.922 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:21.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:22.014 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:22.012+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:22.075 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:22.076 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:22.084 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:22.126 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:22.124+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:22.210 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:22.207+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:22.241 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:22.334 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:22.332+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:22.397 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:22.397 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:22.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:22.397 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:22.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:30 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778566+0000", 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6156852530000005, 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079976379999999996, 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 30 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.398 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778566+0000", 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778566+0000", 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778567+0000", 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:19:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778576+0000", 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8300000000000002e-06 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778584+0000", 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9489999999999996e-06 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778588+0000", 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1899999999999997e-06 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778635+0000", 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7098000000000002e-05 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778722+0000", 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7299000000000006e-05 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778874+0000", 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00015163700000000001 2024-05-08T03:19:22.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780936+0000", 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020618659999999999 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786504+0000", 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0055681180000000004 2024-05-08T03:19:22.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786538+0000", 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3942999999999997e-05 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786564+0000", 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6177e-05 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.403 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:31 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778604+0000", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6156478629999995, 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080424410000000009, 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 31 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778604+0000", 2024-05-08T03:19:22.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778604+0000", 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778604+0000", 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.82e-07 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778608+0000", 2024-05-08T03:19:22.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.878e-06 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778614+0000", 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3640000000000004e-06 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778620+0000", 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7180000000000001e-06 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779174+0000", 2024-05-08T03:19:22.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00055373900000000003 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779221+0000", 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7157000000000003e-05 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779299+0000", 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8059999999999995e-05 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782273+0000", 2024-05-08T03:19:22.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002974024 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786595+0000", 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0043217209999999997 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786625+0000", 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0063999999999999e-05 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786646+0000", 2024-05-08T03:19:22.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1234e-05 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:32 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778646+0000", 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6156057029999999, 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081122899999999994, 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.409 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 32 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778646+0000", 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778646+0000", 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778647+0000", 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1900000000000002e-07 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778718+0000", 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1540999999999998e-05 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778741+0000", 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2506000000000001e-05 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778747+0000", 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6760000000000001e-06 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779552+0000", 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080518199999999999 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779600+0000", 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7911999999999998e-05 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779684+0000", 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4107e-05 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782666+0000", 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0029815419999999998 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786693+0000", 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040271359999999997 2024-05-08T03:19:22.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786727+0000", 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4295999999999998e-05 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786758+0000", 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0772999999999998e-05 2024-05-08T03:19:22.414 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:33 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778766+0000", 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6154854010000008, 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080815729999999999, 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.415 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 33 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778766+0000", 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778766+0000", 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778767+0000", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1699999999999997e-07 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778772+0000", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.933e-06 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778792+0000", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599e-05 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778801+0000", 2024-05-08T03:19:22.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3659999999999992e-06 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780150+0000", 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013488479999999999 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780199+0000", 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9041e-05 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780276+0000", 2024-05-08T03:19:22.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7683999999999997e-05 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782678+0000", 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0024017219999999998 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786798+0000", 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041201709999999997 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786826+0000", 2024-05-08T03:19:22.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.73e-05 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786848+0000", 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2192e-05 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:34 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778813+0000", 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6154384480000008, 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081237759999999992, 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.420 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 34 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778813+0000", 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778813+0000", 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778814+0000", 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778818+0000", 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4469999999999999e-06 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778826+0000", 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3930000000000002e-06 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778833+0000", 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5580000000000003e-06 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780546+0000", 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017129999999999999 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780597+0000", 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000001e-05 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780688+0000", 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0592000000000003e-05 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782681+0000", 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0019933659999999999 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786882+0000", 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042011030000000003 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786911+0000", 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.879e-05 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786937+0000", 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6072999999999999e-05 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.425 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:35 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778845+0000", 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6154068069999994, 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081738599999999998, 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 35 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.426 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778845+0000", 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778845+0000", 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778845+0000", 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.44e-07 2024-05-08T03:19:22.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778936+0000", 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0333000000000004e-05 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778944+0000", 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778951+0000", 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9990000000000002e-06 2024-05-08T03:19:22.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.780956+0000", 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020051309999999998 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781010+0000", 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3557000000000002e-05 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781090+0000", 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0100999999999998e-05 2024-05-08T03:19:22.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782684+0000", 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001594549 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786971+0000", 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00428656 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.786997+0000", 2024-05-08T03:19:22.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6054000000000002e-05 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.787019+0000", 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1611e-05 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:36 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.778970+0000", 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6152821189999997, 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081326939999999993, 2024-05-08T03:19:22.431 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 36 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778970+0000", 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778970+0000", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778970+0000", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6899999999999996e-07 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778976+0000", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4099999999999999e-06 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778984+0000", 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6999999999999997e-06 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.778990+0000", 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9900000000000002e-06 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781495+0000", 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025050599999999999 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781556+0000", 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0217000000000003e-05 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781641+0000", 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5735000000000005e-05 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782696+0000", 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001054368 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.787047+0000", 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0043514770000000003 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.787076+0000", 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8328000000000001e-05 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.787102+0000", 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6639999999999999e-05 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.436 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:37 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.779001+0000", 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.6152503570000007, 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0091832909999999997, 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.437 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 37 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779001+0000", 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779001+0000", 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779002+0000", 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3600000000000004e-07 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779006+0000", 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1509999999999997e-06 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779013+0000", 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.779018+0000", 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3070000000000002e-06 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781897+0000", 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028791559999999999 2024-05-08T03:19:22.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.781949+0000", 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1656000000000001e-05 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782071+0000", 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000121846 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.782698+0000", 2024-05-08T03:19:22.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00062729400000000003 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.788056+0000", 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0053574369999999996 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.788130+0000", 2024-05-08T03:19:22.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.788185+0000", 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4876000000000001e-05 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:56 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.814587+0000", 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.5796642740000006, 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082877739999999995, 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.443 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 56 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.814587+0000", 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.814587+0000", 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.814588+0000", 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9200000000000002e-07 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.814590+0000", 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1859999999999999e-06 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.814593+0000", 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4640000000000002e-06 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.814596+0000", 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5670000000000002e-06 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.818050+0000", 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003454052 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.818097+0000", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6903000000000001e-05 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.818173+0000", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6422999999999997e-05 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.818736+0000", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00056253599999999998 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.822818+0000", 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004082291 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.822850+0000", 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1887999999999997e-05 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.822875+0000", 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5072e-05 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.841150+0000", 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.553101968, 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079302469999999996, 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 69 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841150+0000", 2024-05-08T03:19:22.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841150+0000", 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841150+0000", 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7700000000000001e-07 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841186+0000", 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6495000000000002e-05 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841201+0000", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4472e-05 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841206+0000", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5829999999999997e-06 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841966+0000", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00075937599999999997 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.842013+0000", 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6779000000000001e-05 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.842090+0000", 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7404999999999995e-05 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.842990+0000", 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000900449 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849020+0000", 2024-05-08T03:19:22.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0060293990000000004 2024-05-08T03:19:22.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849055+0000", 2024-05-08T03:19:22.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5371999999999999e-05 2024-05-08T03:19:22.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849080+0000", 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4640000000000001e-05 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.841215+0000", 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.5530369840000002, 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079640809999999996, 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.458 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 70 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841215+0000", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841215+0000", 2024-05-08T03:19:22.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841215+0000", 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.03e-07 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841218+0000", 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841223+0000", 2024-05-08T03:19:22.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9989999999999999e-06 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.841226+0000", 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7690000000000001e-06 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.842631+0000", 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014050379999999999 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.842677+0000", 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6239000000000002e-05 2024-05-08T03:19:22.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.842750+0000", 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2540999999999995e-05 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.843244+0000", 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049427399999999995 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849113+0000", 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058687519999999996 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849143+0000", 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9997000000000001e-05 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849179+0000", 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5992999999999999e-05 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:72 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.849718+0000", 2024-05-08T03:19:22.463 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.5445339960000002, 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078799060000000008, 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 72 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849718+0000", 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849718+0000", 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849718+0000", 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0999999999999998e-07 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849775+0000", 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6598999999999998e-05 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849782+0000", 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3950000000000003e-06 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849787+0000", 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1970000000000004e-06 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.850276+0000", 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000488152 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.850333+0000", 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7485999999999998e-05 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.850414+0000", 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0464999999999996e-05 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.853824+0000", 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0034104140000000001 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857535+0000", 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037108179999999998 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857571+0000", 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5907e-05 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857598+0000", 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6863e-05 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.468 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:73 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.849802+0000", 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.5444499379999996, 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079039409999999994, 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 73 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849802+0000", 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849802+0000", 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849802+0000", 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6199999999999998e-07 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849856+0000", 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3579000000000003e-05 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849875+0000", 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8783e-05 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849881+0000", 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.827e-06 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.850713+0000", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00083160600000000003 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.850769+0000", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5958000000000002e-05 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.850843+0000", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3870999999999998e-05 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.853869+0000", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030262879999999998 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857639+0000", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00376984 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857670+0000", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1065999999999997e-05 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857706+0000", 2024-05-08T03:19:22.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5561000000000001e-05 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:74 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:13.849895+0000", 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.5443564989999992, 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.007887603, 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.474 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 74 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849895+0000", 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849895+0000", 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849896+0000", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.06e-07 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849900+0000", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7360000000000001e-06 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849907+0000", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8469999999999998e-06 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.849917+0000", 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9140000000000003e-06 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.853578+0000", 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0036605019999999999 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.853623+0000", 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5204000000000003e-05 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.853693+0000", 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0151e-05 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.854124+0000", 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00043120000000000002 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857733+0000", 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0036091999999999999 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857761+0000", 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7470999999999998e-05 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:13.857783+0000", 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1772000000000001e-05 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:14.078989+0000", 2024-05-08T03:19:22.479 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.3152625180000008, 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010691767, 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 113 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.078989+0000", 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.078989+0000", 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.078990+0000", 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.078996+0000", 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079003+0000", 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4850000000000003e-06 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079008+0000", 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0939999999999998e-06 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.082332+0000", 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033239939999999998 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.082411+0000", 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8750000000000003e-05 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.082530+0000", 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000118756 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084180+0000", 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016508110000000001 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.089549+0000", 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0053690830000000002 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.089624+0000", 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4442000000000003e-05 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.089681+0000", 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6938999999999998e-05 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.484 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:14.079233+0000", 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.3150191099999997, 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010624536, 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 114 2024-05-08T03:19:22.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079233+0000", 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079233+0000", 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079233+0000", 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8999999999999996e-07 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079237+0000", 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1470000000000003e-06 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079245+0000", 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8599999999999993e-06 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079250+0000", 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5120000000000002e-06 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.082811+0000", 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035612909999999998 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.082856+0000", 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5076e-05 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.082943+0000", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6670999999999997e-05 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084605+0000", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001661959 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.089764+0000", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051588650000000003 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.089816+0000", 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2771e-05 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.089857+0000", 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0694e-05 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:14.079418+0000", 2024-05-08T03:19:22.490 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.3148336539999992, 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019542904999999999, 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 115 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079418+0000", 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079418+0000", 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079419+0000", 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4199999999999996e-07 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079507+0000", 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8072000000000007e-05 2024-05-08T03:19:22.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079518+0000", 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1695000000000001e-05 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079532+0000", 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3386e-05 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.083197+0000", 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003665284 2024-05-08T03:19:22.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.083271+0000", 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3733999999999996e-05 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.083389+0000", 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000118068 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084608+0000", 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012189849999999999 2024-05-08T03:19:22.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.098868+0000", 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014260063999999999 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.098923+0000", 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4830999999999997e-05 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.098961+0000", 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8244000000000002e-05 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.495 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:14.079550+0000", 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.3147016790000006, 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019506577000000001, 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 116 2024-05-08T03:19:22.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079550+0000", 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079550+0000", 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079551+0000", 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4399999999999999e-07 2024-05-08T03:19:22.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079556+0000", 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1250000000000001e-06 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079564+0000", 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0290000000000005e-06 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079569+0000", 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5829999999999997e-06 2024-05-08T03:19:22.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.083844+0000", 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042746900000000003 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.083895+0000", 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0671999999999999e-05 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.083966+0000", 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1254000000000006e-05 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084611+0000", 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00064448299999999995 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.099002+0000", 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014390994000000001 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.099031+0000", 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9612999999999999e-05 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.099057+0000", 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5389999999999999e-05 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:14.079686+0000", 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.3145660899999996, 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019492538, 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.501 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 117 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079686+0000", 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079686+0000", 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079686+0000", 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3900000000000004e-07 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079696+0000", 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0040999999999999e-05 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079704+0000", 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1580000000000005e-06 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.079712+0000", 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2010000000000002e-06 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084206+0000", 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0044944700000000004 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084284+0000", 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7674000000000002e-05 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084416+0000", 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013207900000000001 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.084770+0000", 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035457499999999999 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.099093+0000", 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014322962 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.099142+0000", 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9004e-05 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.099178+0000", 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5635e-05 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4206.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T03:19:22.506 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:14.691007+0000", 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7032443690000001, 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084839909999999998, 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4206", 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/2600038353", 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 141 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.691007+0000", 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.691007+0000", 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.691008+0000", 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3599999999999999e-07 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.691014+0000", 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1789999999999996e-06 2024-05-08T03:19:22.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.691018+0000", 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.691022+0000", 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4630000000000001e-06 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.697314+0000", 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062917629999999997 2024-05-08T03:19:22.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.697355+0000", 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1625999999999998e-05 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.697435+0000", 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9562999999999997e-05 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.697866+0000", 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00043092500000000003 2024-05-08T03:19:22.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.699448+0000", 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015823079999999999 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.699473+0000", 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4674000000000001e-05 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:14.699491+0000", 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8566000000000001e-05 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.511 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.512 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.512 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:22.512 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:22.512 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:22.512 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:22.449+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:22.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:22.557 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:22.556+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:22.618 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:22.618 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:22.618 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:22.618 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:22.626 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:22.668 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:22.666+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:22.754 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:22.753+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:22.773 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:22.773 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:22.773 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:22.773 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:22.773 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:22.780 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:19:22.785 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:22.855 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:22.853+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:22.895 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [] 2024-05-08T03:19:22.896 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:22.905 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:22.989 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:22.987+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:23.058 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:23.056+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:23.062 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:23.158 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:23.157+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:23.208 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:23.209 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:23.209 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:23.209 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:23.209 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:23.218 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:23.259 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:23.257+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:23.361 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:23.359+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:23.367 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:23.367 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:23.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:23.367 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:23.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.367 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:20 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.757281+0000", 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.6073746100000008, 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0065633569999999997, 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757281+0000", 2024-05-08T03:19:23.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757281+0000", 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757283+0000", 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4330000000000001e-06 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757539+0000", 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025649100000000002 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757562+0000", 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2965000000000001e-05 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757575+0000", 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3179999999999999e-05 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757644+0000", 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8690999999999997e-05 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757668+0000", 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3977000000000001e-05 2024-05-08T03:19:23.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763783+0000", 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0061144149999999998 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763845+0000", 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2205000000000005e-05 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:21 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.758060+0000", 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.6065959769999996, 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0058140400000000004, 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758060+0000", 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758060+0000", 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758061+0000", 2024-05-08T03:19:23.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8999999999999996e-07 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758120+0000", 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9466000000000003e-05 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758130+0000", 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9939999999999995e-06 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758138+0000", 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.011e-06 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758188+0000", 2024-05-08T03:19:23.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0312000000000002e-05 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758208+0000", 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9794000000000001e-05 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763849+0000", 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0056411029999999997 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763874+0000", 2024-05-08T03:19:23.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4769999999999998e-05 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:30 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'16, mlcod=17'14)", 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.779003+0000", 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5856528599999997, 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0057940429999999996, 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.378 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779003+0000", 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779003+0000", 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779005+0000", 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.68e-06 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779018+0000", 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3192e-05 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779040+0000", 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2092000000000001e-05 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779052+0000", 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2014999999999999e-05 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779131+0000", 2024-05-08T03:19:23.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9332000000000005e-05 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779156+0000", 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4567e-05 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.784676+0000", 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0055198449999999998 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.784797+0000", 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012132 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:31 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'17, mlcod=17'14)", 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5852123060000007, 2024-05-08T03:19:23.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063259120000000004, 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7899999999999999e-07 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779450+0000", 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7490000000000004e-06 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779459+0000", 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4029999999999996e-06 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779466+0000", 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4160000000000001e-06 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779512+0000", 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5731e-05 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779529+0000", 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7220000000000001e-05 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.784801+0000", 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0052718189999999996 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.785770+0000", 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000969095 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.387 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:32 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.779809+0000", 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5848465839999992, 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063913019999999997, 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779809+0000", 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779809+0000", 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779810+0000", 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6599999999999996e-07 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779866+0000", 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6156000000000002e-05 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779878+0000", 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.183e-05 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779885+0000", 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8730000000000001e-06 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779940+0000", 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5047000000000003e-05 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779956+0000", 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6427999999999999e-05 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.785772+0000", 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0058157809999999999 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786201+0000", 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042852100000000001 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:33 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'19, mlcod=17'14)", 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.780398+0000", 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5842576519999998, 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.005828031, 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.392 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780398+0000", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780398+0000", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780399+0000", 2024-05-08T03:19:23.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0699999999999997e-07 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780406+0000", 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8000000000000001e-06 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780414+0000", 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1149999999999994e-06 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780420+0000", 2024-05-08T03:19:23.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7529999999999998e-06 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780463+0000", 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3742999999999998e-05 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780479+0000", 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6116e-05 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786203+0000", 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.005723672 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786226+0000", 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3325000000000001e-05 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:34 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'20, mlcod=17'14)", 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.780792+0000", 2024-05-08T03:19:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5838636610000005, 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0054587849999999999, 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780792+0000", 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780792+0000", 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780793+0000", 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0100000000000005e-07 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780798+0000", 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9740000000000001e-06 2024-05-08T03:19:23.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780807+0000", 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8340000000000005e-06 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780813+0000", 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0680000000000002e-06 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780869+0000", 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6026000000000002e-05 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780886+0000", 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7130999999999999e-05 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786228+0000", 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0053418759999999997 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786251+0000", 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3374999999999999e-05 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:37 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'14)", 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5824822869999995, 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0056224350000000003, 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7999999999999995e-07 2024-05-08T03:19:23.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782179+0000", 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8690000000000003e-06 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782189+0000", 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3850000000000008e-06 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782195+0000", 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782253+0000", 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8758000000000002e-05 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782270+0000", 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6784000000000001e-05 2024-05-08T03:19:23.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.787742+0000", 2024-05-08T03:19:23.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0054716820000000003 2024-05-08T03:19:23.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.787796+0000", 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4423999999999999e-05 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:69 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'39)", 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5221362680000006, 2024-05-08T03:19:23.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0061802039999999999, 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:23.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5000000000000002e-07 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842554+0000", 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3872999999999999e-05 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842566+0000", 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2089999999999999e-05 2024-05-08T03:19:23.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842573+0000", 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5069999999999997e-06 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842640+0000", 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6698000000000004e-05 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842661+0000", 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1345e-05 2024-05-08T03:19:23.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848635+0000", 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0059744990000000003 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848700+0000", 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4542000000000006e-05 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:70 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'39)", 2024-05-08T03:19:23.412 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5214092269999995, 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0054836540000000001, 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9299999999999998e-07 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843253+0000", 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3890000000000004e-06 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843264+0000", 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1712000000000001e-05 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843271+0000", 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3779999999999998e-06 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843339+0000", 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7753e-05 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843363+0000", 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4422000000000001e-05 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848703+0000", 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0053395889999999996 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848730+0000", 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7818e-05 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:72 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'46, mlcod=17'43)", 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.850881+0000", 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5137747879999992, 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063445560000000003, 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.417 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850881+0000", 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850881+0000", 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850882+0000", 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.015e-06 2024-05-08T03:19:23.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850985+0000", 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010293099999999999 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851008+0000", 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2881999999999999e-05 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851016+0000", 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4340000000000003e-06 2024-05-08T03:19:23.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851178+0000", 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016234100000000001 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851203+0000", 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4779000000000001e-05 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857164+0000", 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.005961002 2024-05-08T03:19:23.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857226+0000", 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2172e-05 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:73 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'47, mlcod=17'43)", 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5136230079999997, 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062212869999999998, 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.421 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.422 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.422 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851095+0000", 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1148999999999993e-05 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851106+0000", 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1336e-05 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851113+0000", 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0720000000000002e-06 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851600+0000", 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00048737100000000001 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851620+0000", 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9660000000000002e-05 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857229+0000", 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0056093189999999998 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857254+0000", 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4975999999999999e-05 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:105 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'64)", 2024-05-08T03:19:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2865794069999996, 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.006530821, 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4499999999999997e-07 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078083+0000", 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078097+0000", 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4508999999999999e-05 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078104+0000", 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3829999999999997e-06 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078183+0000", 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9265999999999996e-05 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078198+0000", 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.503e-05 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084567+0000", 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063695669999999996 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084607+0000", 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9950999999999997e-05 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:106 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'64)", 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.078561+0000", 2024-05-08T03:19:23.431 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2860950229999997, 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0060741110000000001, 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078561+0000", 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078561+0000", 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078562+0000", 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3099999999999997e-07 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078613+0000", 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1532000000000001e-05 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078627+0000", 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4116e-05 2024-05-08T03:19:23.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078634+0000", 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2310000000000001e-06 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078691+0000", 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7297999999999999e-05 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078708+0000", 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6920999999999999e-05 2024-05-08T03:19:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084610+0000", 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0059019540000000001 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084635+0000", 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5428000000000001e-05 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.435 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:108 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'70, mlcod=17'64)", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.079739+0000", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2849167829999999, 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0059717759999999998, 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079739+0000", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079739+0000", 2024-05-08T03:19:23.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079740+0000", 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5700000000000002e-07 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079745+0000", 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5049999999999998e-06 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079758+0000", 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2697000000000001e-05 2024-05-08T03:19:23.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079764+0000", 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3740000000000003e-06 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.080136+0000", 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00037181099999999999 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.080156+0000", 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0244000000000001e-05 2024-05-08T03:19:23.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085668+0000", 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0055119979999999997 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085711+0000", 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2589999999999997e-05 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.439 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:113 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'75, mlcod=17'65)", 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.082974+0000", 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2816817730000007, 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062840980000000001, 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082974+0000", 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082974+0000", 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082975+0000", 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6599999999999996e-07 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082983+0000", 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1300000000000001e-06 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082993+0000", 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0128e-05 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083000+0000", 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6529999999999997e-06 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083104+0000", 2024-05-08T03:19:23.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010425 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083122+0000", 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8016000000000002e-05 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089197+0000", 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0060752660000000002 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089258+0000", 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0989000000000002e-05 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:114 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'76, mlcod=17'65)", 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2816240440000009, 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.006255783, 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0800000000000001e-07 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083037+0000", 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5820000000000003e-06 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083044+0000", 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5070000000000003e-06 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083049+0000", 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8269999999999997e-06 2024-05-08T03:19:23.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084314+0000", 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0012644589999999999 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084333+0000", 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9287e-05 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089262+0000", 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0049288439999999999 2024-05-08T03:19:23.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089288+0000", 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5969000000000002e-05 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:115 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'65)", 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2807095319999995, 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014688414, 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.448 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083947+0000", 2024-05-08T03:19:23.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1900000000000001e-07 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083985+0000", 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7769999999999999e-05 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084001+0000", 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.592e-05 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084009+0000", 2024-05-08T03:19:23.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9109999999999993e-06 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084686+0000", 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00067690100000000004 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084703+0000", 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.755e-05 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098569+0000", 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013865446999999999 2024-05-08T03:19:23.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098635+0000", 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6095999999999999e-05 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:116 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'65)", 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2805166630000002, 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014525575000000001, 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.452 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084140+0000", 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1900000000000003e-07 2024-05-08T03:19:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084144+0000", 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6149999999999999e-06 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084153+0000", 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6459999999999999e-06 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084159+0000", 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0079999999999999e-06 2024-05-08T03:19:23.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085434+0000", 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001275263 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085455+0000", 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0893000000000001e-05 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098638+0000", 2024-05-08T03:19:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013182523 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098665+0000", 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7107999999999999e-05 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:117 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'65)", 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.2798486160000007, 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013907645, 2024-05-08T03:19:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:23.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084808+0000", 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-07 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084813+0000", 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2259999999999996e-06 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084822+0000", 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5830000000000007e-06 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:23.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084830+0000", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9650000000000008e-06 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085897+0000", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001067496 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085917+0000", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0080000000000001e-05 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098667+0000", 2024-05-08T03:19:23.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012749462 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098715+0000", 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8285999999999998e-05 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:23.460 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:23.461 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:23.462 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:23.459+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:23.562 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:23.561+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:23.569 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:23.569 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:23.579 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:23.617 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:23.663 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:23.662+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:23.729 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:23.729 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:23.729 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:23.729 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:23.729 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:23.738 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:23.745 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:23.745 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:23.764 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:23.763+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:23.765 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:23.765 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:23.785 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:23.785 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:23.802 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:23.802 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:23.820 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:23.820 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:23.838 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:23.838 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:23.853 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:23.854 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:23.856 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:23.856 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:23.863 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:23.874 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:23.874 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:23.875 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:23.872+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:23.966 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:23.964+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:23.981 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:23.981 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:23.990 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:24.067 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:24.066+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:24.137 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.137 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.147 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:24.190 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:24.189+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:24.255 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.255 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.264 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:24.305 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:24.304+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:24.370 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:24.368+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:24.382 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.382 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:24.415 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:24.415 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:24.416 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-05-08T03:19:24.416 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T03:19:24.416 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T03:19:24.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-05-08T03:19:24.477 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:24.475+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:24.523 INFO:teuthology.orchestra.run.smithi177.stdout:1 writesame 0x7b4aeb7 thru 0x7b59ea3 data_size 0x1(0xefed bytes) 2024-05-08T03:19:24.524 INFO:teuthology.orchestra.run.smithi177.stdout:3 punch from 0x178da44 to 0x1797a5c, (0xa018 bytes) 2024-05-08T03:19:24.524 INFO:teuthology.orchestra.run.smithi177.stdout:4 read 0x59f5e0d thru 0x59facd4 (0x4ec8 bytes) 2024-05-08T03:19:24.524 INFO:teuthology.orchestra.run.smithi177.stdout:5 punch from 0x77b5a0d to 0x77c3ac5, (0xe0b8 bytes) 2024-05-08T03:19:24.524 INFO:teuthology.orchestra.run.smithi177.stdout:6 punch from 0x61f5259 to 0x620215d, (0xcf04 bytes) 2024-05-08T03:19:24.531 INFO:teuthology.orchestra.run.smithi177.stdout:7 write 0x890e021 thru 0x8917db6 (0x9d96 bytes) 2024-05-08T03:19:24.545 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:24.545 INFO:teuthology.orchestra.run.smithi188.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T03:19:24.545 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:24.546 INFO:teuthology.orchestra.run.smithi177.stdout:8 write 0xb5096e5 thru 0xb50f623 (0x5f3f bytes) 2024-05-08T03:19:24.547 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.547 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.554 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.554 INFO:teuthology.orchestra.run.smithi177.stdout:10 write 0xaa7fbd6 thru 0xaa85a87 (0x5eb2 bytes) 2024-05-08T03:19:24.558 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:24.559 INFO:teuthology.orchestra.run.smithi177.stdout:11 read 0x36df069 thru 0x36e0567 (0x14ff bytes) 2024-05-08T03:19:24.559 INFO:teuthology.orchestra.run.smithi177.stdout:12 read 0x243af9f thru 0x2446246 (0xb2a8 bytes) 2024-05-08T03:19:24.559 INFO:teuthology.orchestra.run.smithi177.stdout:13 write 0x420a5a3 thru 0x420b70f (0x116d bytes) 2024-05-08T03:19:24.563 INFO:teuthology.orchestra.run.smithi177.stdout:14 write 0x179c6b2 thru 0x17a8fc2 (0xc911 bytes) 2024-05-08T03:19:24.566 INFO:teuthology.orchestra.run.smithi177.stdout:16 read 0x7970ed0 thru 0x79763e5 (0x5516 bytes) 2024-05-08T03:19:24.567 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xb4ae508 2024-05-08T03:19:24.567 INFO:teuthology.orchestra.run.smithi177.stdout:17 trunc from 0xb50f624 to 0xb4ae508 2024-05-08T03:19:24.572 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:24.570+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:24.574 INFO:teuthology.orchestra.run.smithi177.stdout:18 read 0x29daa36 thru 0x29e1920 (0x6eeb bytes) 2024-05-08T03:19:24.574 INFO:teuthology.orchestra.run.smithi177.stdout:20 read 0xab7c2f4 thru 0xab7d4f2 (0x11ff bytes) 2024-05-08T03:19:24.582 INFO:teuthology.orchestra.run.smithi177.stdout:21 write 0xc8820d6 thru 0xc883e0b (0x1d36 bytes) 2024-05-08T03:19:24.587 INFO:teuthology.orchestra.run.smithi177.stdout:22 read 0x1a1bade thru 0x1a282c9 (0xc7ec bytes) 2024-05-08T03:19:24.587 INFO:teuthology.orchestra.run.smithi177.stdout:25 read 0x56237b6 thru 0x562569f (0x1eea bytes) 2024-05-08T03:19:24.587 INFO:teuthology.orchestra.run.smithi177.stdout:26 read 0x8d7ea54 thru 0x8d8104d (0x25fa bytes) 2024-05-08T03:19:24.587 INFO:teuthology.orchestra.run.smithi177.stdout:27 read 0x27a559e thru 0x27b3a7a (0xe4dd bytes) 2024-05-08T03:19:24.587 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.587 INFO:teuthology.orchestra.run.smithi177.stdout:30 writesame 0x513fe81 thru 0x514baec data_size 0x3e(0xbc6c bytes) 2024-05-08T03:19:24.591 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xd3c1996 2024-05-08T03:19:24.591 INFO:teuthology.orchestra.run.smithi177.stdout:32 trunc from 0xc883e0c to 0xd3c1996 2024-05-08T03:19:24.598 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.598 INFO:teuthology.orchestra.run.smithi177.stdout:34 read 0x15c1aeb thru 0x15c4197 (0x26ad bytes) 2024-05-08T03:19:24.598 INFO:teuthology.orchestra.run.smithi177.stdout:35 write 0xad2c1f9 thru 0xad35134 (0x8f3c bytes) 2024-05-08T03:19:24.602 INFO:teuthology.orchestra.run.smithi177.stdout:36 write 0x1223873 thru 0x1230167 (0xc8f5 bytes) 2024-05-08T03:19:24.607 INFO:teuthology.orchestra.run.smithi177.stdout:37 read 0x2e8d898 thru 0x2e9490d (0x7076 bytes) 2024-05-08T03:19:24.607 INFO:teuthology.orchestra.run.smithi177.stdout:38 writesame 0x1b7a9ba thru 0x1b80f9c data_size 0x1(0x65e3 bytes) 2024-05-08T03:19:24.623 INFO:teuthology.orchestra.run.smithi177.stdout:40 read 0xfda28d thru 0xfde476 (0x41ea bytes) 2024-05-08T03:19:24.638 INFO:teuthology.orchestra.run.smithi177.stdout:41 trunc from 0xd3c1996 to 0x40b2bc 2024-05-08T03:19:24.656 INFO:teuthology.orchestra.run.smithi177.stdout:42 read 0x29ac18 thru 0x2a7491 (0xc87a bytes) 2024-05-08T03:19:24.669 INFO:teuthology.orchestra.run.smithi177.stdout:43 writesame 0x496cb88 thru 0x4973394 data_size 0xd(0x680d bytes) 2024-05-08T03:19:24.669 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.669 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xe4d010b 2024-05-08T03:19:24.673 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:24.671+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:24.681 INFO:teuthology.orchestra.run.smithi177.stdout:45 trunc from 0x4973395 to 0xe4d010b 2024-05-08T03:19:24.689 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.689 INFO:teuthology.orchestra.run.smithi177.stdout:47 write 0xa0ccc7f thru 0xa0ce11d (0x149f bytes) 2024-05-08T03:19:24.692 INFO:teuthology.orchestra.run.smithi177.stdout:49 writesame 0x94d89a thru 0x9579fb data_size 0x5b(0xa162 bytes) 2024-05-08T03:19:24.696 INFO:teuthology.orchestra.run.smithi177.stdout:50 read 0x1a6aff1 thru 0x1a79cda (0xecea bytes) 2024-05-08T03:19:24.697 INFO:teuthology.orchestra.run.smithi177.stdout:52 write 0x7d0a29f thru 0x7d13df4 (0x9b56 bytes) 2024-05-08T03:19:24.699 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.705 INFO:teuthology.orchestra.run.smithi177.stdout:58 writesame 0xea74270 thru 0xea78b30 data_size 0x5(0x48c1 bytes) 2024-05-08T03:19:24.705 INFO:teuthology.orchestra.run.smithi177.stdout:59 writesame 0xe9b6ec7 thru 0xe9be69e data_size 0x34(0x77d8 bytes) 2024-05-08T03:19:24.709 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.710 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.711 INFO:teuthology.orchestra.run.smithi177.stdout:61 trunc from 0xea78b31 to 0xc1faa5a 2024-05-08T03:19:24.720 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:24.726 INFO:teuthology.orchestra.run.smithi177.stdout:62 trunc from 0xc1faa5a to 0x891a3c6 2024-05-08T03:19:24.736 INFO:teuthology.orchestra.run.smithi177.stdout:63 write 0x611336c thru 0x611d811 (0xa4a6 bytes) 2024-05-08T03:19:24.741 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:24.753 INFO:teuthology.orchestra.run.smithi177.stdout:66 writesame 0xe0fac07 thru 0xe0ffb0c data_size 0x46(0x4f06 bytes) 2024-05-08T03:19:24.754 INFO:teuthology.orchestra.run.smithi177.stdout:67 read 0x2f7bb7d thru 0x2f8a010 (0xe494 bytes) 2024-05-08T03:19:24.754 INFO:teuthology.orchestra.run.smithi177.stdout:68 write 0x408142d thru 0x4083a48 (0x261c bytes) 2024-05-08T03:19:24.757 INFO:teuthology.orchestra.run.smithi177.stdout:71 writesame 0xc901e4 thru 0xc98755 data_size 0x3a(0x8572 bytes) 2024-05-08T03:19:24.760 INFO:teuthology.orchestra.run.smithi177.stdout:73 write 0x30997c5 thru 0x309dbd6 (0x4412 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:76 read 0xd2550a7 thru 0xd2637b9 (0xe713 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:77 punch from 0x1f0ad60 to 0x1f12bf9, (0x7e99 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:79 read 0xa760de5 thru 0xa769401 (0x861d bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:80 read 0xbfde968 thru 0xbfe381d (0x4eb6 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:83 read 0x8001246 thru 0x8007266 (0x6021 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:84 punch from 0x4a051a2 to 0x4a11d92, (0xcbf0 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:86 punch from 0x7ab5088 to 0x7abeceb, (0x9c63 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:89 punch from 0x30e47a3 to 0x30e9dc4, (0x5621 bytes) 2024-05-08T03:19:24.764 INFO:teuthology.orchestra.run.smithi177.stdout:90 read 0x946f4d5 thru 0x947405a (0x4b86 bytes) 2024-05-08T03:19:24.773 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:24.772+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:24.776 INFO:teuthology.orchestra.run.smithi177.stdout:91 trunc from 0xe0ffb0d to 0x367ad80 2024-05-08T03:19:24.788 INFO:teuthology.orchestra.run.smithi177.stdout:94 read 0xcb85ec thru 0xcb967c (0x1091 bytes) 2024-05-08T03:19:24.796 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:19:24.799 INFO:teuthology.orchestra.run.smithi177.stdout:95 trunc from 0x367ad80 to 0xc92bfab 2024-05-08T03:19:24.805 INFO:teuthology.orchestra.run.smithi177.stdout:97 punch from 0x7489845 to 0x7496703, (0xcebe bytes) 2024-05-08T03:19:24.805 INFO:teuthology.orchestra.run.smithi177.stdout:98 read 0x2e169ed thru 0x2e194c8 (0x2adc bytes) 2024-05-08T03:19:24.805 INFO:teuthology.orchestra.run.smithi177.stdout:99 write 0x1f0a235 thru 0x1f16ef6 (0xccc2 bytes) 2024-05-08T03:19:24.810 INFO:teuthology.orchestra.run.smithi177.stdout:101 read 0x44590e3 thru 0x44644c4 (0xb3e2 bytes) 2024-05-08T03:19:24.811 INFO:teuthology.orchestra.run.smithi177.stdout:103 punch from 0xc360504 to 0xc3671c8, (0x6cc4 bytes) 2024-05-08T03:19:24.811 INFO:teuthology.orchestra.run.smithi177.stdout:105 punch from 0x6c7201d to 0x6c78caa, (0x6c8d bytes) 2024-05-08T03:19:24.811 INFO:teuthology.orchestra.run.smithi177.stdout:106 punch from 0x5662f96 to 0x5669e55, (0x6ebf bytes) 2024-05-08T03:19:24.811 INFO:teuthology.orchestra.run.smithi177.stdout:108 clone 1 order 23 su 4194304 sc 9 2024-05-08T03:19:24.831 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.831 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.840 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:24.884 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:24.882+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:24.946 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:24.946 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:24.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:24.976 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:24.974+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:25.077 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:25.075+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:25.100 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.100 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:25.178 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:25.177+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:25.215 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.215 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.224 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:25.279 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:25.278+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:25.328 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.329 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.338 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:25.383 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:25.381+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:25.443 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.443 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.451 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:25.493 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:25.491+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:25.582 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:25.580+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:25.597 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.597 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:25.683 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:25.681+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:25.751 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.751 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:25.789 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:19:25.804 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:25.802+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:25.874 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.874 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.883 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:25.884 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:25.883+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:25.887 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:25.979 INFO:teuthology.orchestra.run.smithi177.stdout:110 punch from 0x74c0a41 to 0x74c47f3, (0x3db2 bytes) 2024-05-08T03:19:25.984 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:25.983+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:25.988 INFO:teuthology.orchestra.run.smithi177.stdout:111 writesame 0xccea34e thru 0xccf40ca data_size 0x59(0x9d7d bytes) 2024-05-08T03:19:25.990 INFO:teuthology.orchestra.run.smithi177.stdout:112 read 0x826d5e thru 0x830e98 (0xa13b bytes) 2024-05-08T03:19:25.991 INFO:teuthology.orchestra.run.smithi177.stdout:113 read 0x37e193b thru 0x37efc8a (0xe350 bytes) 2024-05-08T03:19:25.991 INFO:teuthology.orchestra.run.smithi177.stdout:114 read 0x81f34d5 thru 0x81fff9f (0xcacb bytes) 2024-05-08T03:19:25.991 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xebf5426 2024-05-08T03:19:25.993 INFO:teuthology.orchestra.run.smithi177.stdout:115 trunc from 0xccf40cb to 0xebf5426 2024-05-08T03:19:25.998 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:25.998 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:25.999 INFO:teuthology.orchestra.run.smithi177.stdout:116 write 0xeb1b3dd thru 0xeb1b518 (0x13c bytes) 2024-05-08T03:19:26.002 INFO:teuthology.orchestra.run.smithi177.stdout:117 write 0xd3cf6c8 thru 0xd3d2be5 (0x351e bytes) 2024-05-08T03:19:26.005 INFO:teuthology.orchestra.run.smithi177.stdout:119 punch from 0x983a1cc to 0x98415b0, (0x73e4 bytes) 2024-05-08T03:19:26.005 INFO:teuthology.orchestra.run.smithi177.stdout:120 punch from 0x17df748 to 0x17e9d3a, (0xa5f2 bytes) 2024-05-08T03:19:26.006 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:26.006 INFO:teuthology.orchestra.run.smithi177.stdout:122 write 0x890c4b0 thru 0x8910719 (0x426a bytes) 2024-05-08T03:19:26.009 INFO:teuthology.orchestra.run.smithi177.stdout:123 punch from 0x1732ffb to 0x173c01d, (0x9022 bytes) 2024-05-08T03:19:26.009 INFO:teuthology.orchestra.run.smithi177.stdout:125 writesame 0x9a14906 thru 0x9a1f8f8 data_size 0x1(0xaff3 bytes) 2024-05-08T03:19:26.010 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:26.032 INFO:teuthology.orchestra.run.smithi177.stdout:126 punch from 0x557192b to 0x557deea, (0xc5bf bytes) 2024-05-08T03:19:26.032 INFO:teuthology.orchestra.run.smithi177.stdout:127 read 0x31dff50 thru 0x31e2f64 (0x3015 bytes) 2024-05-08T03:19:26.032 INFO:teuthology.orchestra.run.smithi177.stdout:128 writesame 0x3128d75 thru 0x31363b9 data_size 0x1(0xd645 bytes) 2024-05-08T03:19:26.058 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:26.058 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:26.062 INFO:teuthology.orchestra.run.smithi177.stdout:129 write 0x57fe594 thru 0x580c63d (0xe0aa bytes) 2024-05-08T03:19:26.067 INFO:teuthology.orchestra.run.smithi177.stdout:130 punch from 0xda36450 to 0xda387b3, (0x2363 bytes) 2024-05-08T03:19:26.067 INFO:teuthology.orchestra.run.smithi177.stdout:131 write 0xb4b240 thru 0xb4f930 (0x46f1 bytes) 2024-05-08T03:19:26.071 INFO:teuthology.orchestra.run.smithi177.stdout:132 write 0xc254a36 thru 0xc260ef0 (0xc4bb bytes) 2024-05-08T03:19:26.074 INFO:teuthology.orchestra.run.smithi177.stdout:133 read 0x2cd60f0 thru 0x2cdd73c (0x764d bytes) 2024-05-08T03:19:26.074 INFO:teuthology.orchestra.run.smithi177.stdout:135 punch from 0x82de79b to 0x82e2fe1, (0x4846 bytes) 2024-05-08T03:19:26.074 INFO:teuthology.orchestra.run.smithi177.stdout:136 write 0x36ac62 thru 0x378435 (0xd7d4 bytes) 2024-05-08T03:19:26.079 INFO:teuthology.orchestra.run.smithi177.stdout:137 trunc from 0xebf5426 to 0xd464290 2024-05-08T03:19:26.079 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:26.079 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:26.086 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:26.084+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:26.096 INFO:teuthology.orchestra.run.smithi177.stdout:138 write 0x4f39fcf thru 0x4f3a233 (0x265 bytes) 2024-05-08T03:19:26.099 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:26.099 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:26.101 INFO:teuthology.orchestra.run.smithi177.stdout:139 punch from 0xb659d36 to 0xb669667, (0xf931 bytes) 2024-05-08T03:19:26.101 INFO:teuthology.orchestra.run.smithi177.stdout:141 read 0x7bb65b2 thru 0x7bbb2dd (0x4d2c bytes) 2024-05-08T03:19:26.107 INFO:teuthology.orchestra.run.smithi177.stdout:142 trunc from 0xd464290 to 0x8b04fc9 2024-05-08T03:19:26.122 INFO:teuthology.orchestra.run.smithi177.stdout:143 read 0x1748833 thru 0x17531a0 (0xa96e bytes) 2024-05-08T03:19:26.122 INFO:teuthology.orchestra.run.smithi177.stdout:145 read 0x2d50de2 thru 0x2d5aa4d (0x9c6c bytes) 2024-05-08T03:19:26.128 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:26.128 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:26.128 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:26.128 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:26.128 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:26.133 INFO:teuthology.orchestra.run.smithi177.stdout:146 writesame 0xd9ca225 thru 0xd9d6793 data_size 0x1(0xc56f bytes) 2024-05-08T03:19:26.134 INFO:teuthology.orchestra.run.smithi177.stdout:147 read 0xa675ce2 thru 0xa683739 (0xda58 bytes) 2024-05-08T03:19:26.134 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:26.134 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:26.134 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:26.134 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:26.140 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:26.141 INFO:teuthology.orchestra.run.smithi177.stdout:153 writesame 0xec7671e thru 0xec862d7 data_size 0x7(0xfbba bytes) 2024-05-08T03:19:26.141 INFO:teuthology.orchestra.run.smithi177.stdout:156 punch from 0xc30766c to 0xc309b32, (0x24c6 bytes) 2024-05-08T03:19:26.141 INFO:teuthology.orchestra.run.smithi177.stdout:157 write 0x69a24e8 thru 0x69a483a (0x2353 bytes) 2024-05-08T03:19:26.155 INFO:teuthology.orchestra.run.smithi177.stdout:159 trunc from 0xec862d8 to 0x3c4871c 2024-05-08T03:19:26.173 INFO:teuthology.orchestra.run.smithi177.stdout:161 clone 2 order 24 su 1048576 sc 11 2024-05-08T03:19:26.186 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:26.185+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:26.252 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:22.788537+0000", 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.4615545430000001, 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.003621305, 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11 2024-05-08T03:19:26.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788537+0000", 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788537+0000", 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788539+0000", 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6199999999999999e-06 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788549+0000", 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0121e-05 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788566+0000", 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6789000000000001e-05 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788574+0000", 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4300000000000006e-06 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788630+0000", 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5562000000000001e-05 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.788634+0000", 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2289999999999997e-06 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.790585+0000", 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019509180000000001 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.790716+0000", 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013121699999999999 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.790896+0000", 2024-05-08T03:19:26.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000179877 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.791516+0000", 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00061948999999999999 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792062+0000", 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054640099999999996 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792117+0000", 2024-05-08T03:19:26.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4698999999999999e-05 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792159+0000", 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1952000000000001e-05 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:12 3.3 e22/19 3:c65fd037:::rbd_header.11826496459e:head v 22'1, mlcod=0'0)", 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:22.792700+0000", 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.4573915560000001, 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00060848100000000002, 2024-05-08T03:19:26.259 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792700+0000", 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792700+0000", 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792701+0000", 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8500000000000002e-07 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792704+0000", 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5530000000000001e-06 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792712+0000", 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6210000000000004e-06 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792719+0000", 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5070000000000003e-06 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792766+0000", 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6149000000000001e-05 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.792785+0000", 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9394000000000001e-05 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.793264+0000", 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000479424 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.793309+0000", 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4548000000000001e-05 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:22.797642+0000", 2024-05-08T03:19:26.263 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.4524501619999999, 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00044001199999999999, 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 15 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797642+0000", 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797642+0000", 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797642+0000", 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797646+0000", 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6090000000000001e-06 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797654+0000", 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8120000000000008e-06 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797658+0000", 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.707e-06 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797705+0000", 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6100999999999998e-05 2024-05-08T03:19:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.797781+0000", 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6650999999999995e-05 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.798082+0000", 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030063600000000001 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.267 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:25 3.3 e22/19 3:c65fd037:::rbd_header.11826496459e:head v 22'2, mlcod=0'0)", 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:22.801719+0000", 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.448373202, 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00065311299999999998, 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801719+0000", 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801719+0000", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801720+0000", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3799999999999996e-07 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801725+0000", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3639999999999998e-06 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801734+0000", 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3859999999999996e-06 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801742+0000", 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6320000000000008e-06 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801783+0000", 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0667999999999998e-05 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.801800+0000", 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7033000000000001e-05 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.802321+0000", 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000521401 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:22.802372+0000", 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0691e-05 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.271 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:408 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 3453504~18161] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.069336+0000", 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18075618500000001, 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016354329999999999, 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 408 2024-05-08T03:19:26.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069336+0000", 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069336+0000", 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069338+0000", 2024-05-08T03:19:26.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2290000000000002e-06 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069379+0000", 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0707000000000002e-05 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069394+0000", 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4982e-05 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069400+0000", 2024-05-08T03:19:26.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.579e-06 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069463+0000", 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2917999999999996e-05 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069614+0000", 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015119999999999999 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.069816+0000", 2024-05-08T03:19:26.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020140499999999999 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.070855+0000", 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010387770000000001 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.070914+0000", 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.906e-05 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.070944+0000", 2024-05-08T03:19:26.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0182e-05 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.070971+0000", 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7393999999999999e-05 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:415 3.3 e23/19 3:c52de426:::rbd_data.1182b3dbc99f.000000000000001f:head v 23'29, mlcod=23'27)", 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.081124+0000", 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.168968058, 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00060277900000000001, 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.277 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081124+0000", 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081124+0000", 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081125+0000", 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1340000000000001e-06 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081133+0000", 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5830000000000001e-06 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081146+0000", 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3533000000000001e-05 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081154+0000", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3969999999999997e-06 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081206+0000", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2636000000000002e-05 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081223+0000", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6951000000000001e-05 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081697+0000", 2024-05-08T03:19:26.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047365599999999998 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.081727+0000", 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9889000000000001e-05 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:418 3.1b 3:ddb7123e:::rbd_data.1182b3dbc99f.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4604560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.087346+0000", 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.16274606799999999, 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0012673319999999999, 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.281 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 418 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087346+0000", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087346+0000", 2024-05-08T03:19:26.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087347+0000", 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5379999999999999e-06 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087355+0000", 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0960000000000003e-06 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087369+0000", 2024-05-08T03:19:26.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4805e-05 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087378+0000", 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5480000000000002e-06 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087433+0000", 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4848000000000001e-05 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087533+0000", 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010074899999999999 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.087650+0000", 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000116882 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088096+0000", 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000445759 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088547+0000", 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000450941 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088580+0000", 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2728999999999998e-05 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088613+0000", 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3436999999999999e-05 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:419 3.12 3:4fe66973:::rbd_header.1182b3dbc99f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.088728+0000", 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.16136436800000001, 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091130099999999999, 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 419 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088728+0000", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088728+0000", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088728+0000", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5199999999999997e-07 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088731+0000", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2820000000000001e-06 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088739+0000", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0879999999999997e-06 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088743+0000", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0589999999999996e-06 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088786+0000", 2024-05-08T03:19:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088837+0000", 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1554999999999997e-05 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.088961+0000", 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012383899999999999 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.089354+0000", 2024-05-08T03:19:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039225700000000002 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.089587+0000", 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000233429 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.089615+0000", 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7991e-05 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.089639+0000", 2024-05-08T03:19:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3918999999999999e-05 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:436 3.1b 3:ddb7123e:::rbd_data.1182b3dbc99f.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.109462+0000", 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.140629685, 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00099839599999999992, 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 436 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109462+0000", 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109462+0000", 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109464+0000", 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5689999999999999e-06 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109470+0000", 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6329999999999999e-06 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109485+0000", 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4980000000000001e-05 2024-05-08T03:19:26.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109497+0000", 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1402e-05 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109560+0000", 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3509000000000005e-05 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109615+0000", 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4607000000000001e-05 2024-05-08T03:19:26.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109677+0000", 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1784000000000003e-05 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110163+0000", 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000485873 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110390+0000", 2024-05-08T03:19:26.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00022724 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110426+0000", 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5787999999999997e-05 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110461+0000", 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5011000000000001e-05 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:438 3.3 e23/19 3:c18420c2:::rbd_data.1182b3dbc99f.0000000000000010:head v 23'30, mlcod=23'28)", 2024-05-08T03:19:26.297 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.115290+0000", 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.134802432, 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058440200000000003, 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115290+0000", 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115290+0000", 2024-05-08T03:19:26.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115290+0000", 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9599999999999999e-07 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115297+0000", 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8929999999999999e-06 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115310+0000", 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2717e-05 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115316+0000", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5239999999999997e-06 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115355+0000", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9140000000000001e-05 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115369+0000", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3444e-05 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115849+0000", 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047971 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.115874+0000", 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5278000000000001e-05 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:440 3.12 3:4d5ea9e9:::rbd_data.1182b3dbc99f.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.116435+0000", 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.13365656000000001, 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094580100000000002, 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.301 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 440 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116435+0000", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116435+0000", 2024-05-08T03:19:26.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116436+0000", 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4099999999999997e-07 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116441+0000", 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8289999999999998e-06 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116451+0000", 2024-05-08T03:19:26.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8640000000000008e-06 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116456+0000", 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1410000000000004e-06 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116499+0000", 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2963000000000002e-05 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116583+0000", 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3846999999999999e-05 2024-05-08T03:19:26.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.116690+0000", 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000107534 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117102+0000", 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00041170999999999997 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117313+0000", 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021108699999999999 2024-05-08T03:19:26.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117345+0000", 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1548999999999999e-05 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117381+0000", 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6436000000000001e-05 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:441 3.12 3:4fe66973:::rbd_header.1182b3dbc99f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.117458+0000", 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.13263376900000001, 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010355760000000001, 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 441 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117458+0000", 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117458+0000", 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117459+0000", 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8400000000000004e-07 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117462+0000", 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.456e-06 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117470+0000", 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3440000000000003e-06 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117474+0000", 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3170000000000003e-06 2024-05-08T03:19:26.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117507+0000", 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3135e-05 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117554+0000", 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6854000000000002e-05 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.117676+0000", 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012236699999999999 2024-05-08T03:19:26.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.118045+0000", 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000369012 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.118444+0000", 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039905499999999999 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.118472+0000", 2024-05-08T03:19:26.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7223999999999999e-05 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.118494+0000", 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2228000000000001e-05 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:452 3.12 3:4fe66973:::rbd_header.1182b3dbc99f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.130101+0000", 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.11999082599999999, 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000928117, 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.312 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 452 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130101+0000", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130101+0000", 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130103+0000", 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4750000000000001e-06 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130110+0000", 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1590000000000004e-06 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130125+0000", 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5184999999999999e-05 2024-05-08T03:19:26.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130135+0000", 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0094e-05 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130186+0000", 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1212999999999999e-05 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130242+0000", 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5683999999999997e-05 2024-05-08T03:19:26.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130362+0000", 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000120227 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130776+0000", 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00041372400000000003 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130959+0000", 2024-05-08T03:19:26.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000183182 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.130990+0000", 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0908000000000003e-05 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.131029+0000", 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9266e-05 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:461 3.12 3:4fe66973:::rbd_header.1182b3dbc99f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.136876+0000", 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.11321555699999999, 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001078229, 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 461 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.318 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136876+0000", 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136876+0000", 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136878+0000", 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.595e-06 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136885+0000", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3610000000000003e-06 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136900+0000", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5078e-05 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136907+0000", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9940000000000003e-06 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.136970+0000", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199000000000005e-05 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.137060+0000", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0316999999999996e-05 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.137264+0000", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000204293 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.137817+0000", 2024-05-08T03:19:26.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00055256800000000003 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.137892+0000", 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5683e-05 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.137926+0000", 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3269000000000003e-05 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.137955+0000", 2024-05-08T03:19:26.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8872000000000001e-05 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:469 3.12 3:4d5ea9e9:::rbd_data.1182b3dbc99f.0000000000000011:head [stat,write 1713384~9043] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.140511+0000", 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.109581002, 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011779620000000001, 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.323 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 469 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140511+0000", 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140511+0000", 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140512+0000", 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140520+0000", 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8929999999999994e-06 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140529+0000", 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7690000000000002e-06 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140537+0000", 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6000000000000001e-06 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140574+0000", 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7543999999999998e-05 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140623+0000", 2024-05-08T03:19:26.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8853000000000001e-05 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.140712+0000", 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449000000000006e-05 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.141337+0000", 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00062532199999999999 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.141627+0000", 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00028952099999999999 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.141666+0000", 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9756999999999999e-05 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.141689+0000", 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2614999999999999e-05 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:475 3.12 3:4d5ea9e9:::rbd_data.1182b3dbc99f.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.156499+0000", 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.093592878000000004, 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011365850000000001, 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 475 2024-05-08T03:19:26.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156499+0000", 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156499+0000", 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156501+0000", 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3309999999999998e-06 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156509+0000", 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156524+0000", 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5719999999999999e-05 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156534+0000", 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0059e-05 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156603+0000", 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8126000000000005e-05 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156663+0000", 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0671999999999998e-05 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156736+0000", 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2267000000000005e-05 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.157515+0000", 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00077975400000000002 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.157547+0000", 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2042999999999998e-05 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.157595+0000", 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7759999999999997e-05 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.157636+0000", 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0599999999999998e-05 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.334 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:474 3.3 e23/19 3:c18420c2:::rbd_data.1182b3dbc99f.0000000000000010:head v 23'31, mlcod=23'29)", 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.156796+0000", 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.093295796, 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001492461, 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156796+0000", 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156796+0000", 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156797+0000", 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1799999999999999e-06 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156805+0000", 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1810000000000004e-06 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156815+0000", 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0219e-05 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156821+0000", 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3890000000000002e-06 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156973+0000", 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000152239 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156993+0000", 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9987e-05 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.158248+0000", 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001254968 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.158289+0000", 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0297999999999999e-05 2024-05-08T03:19:26.338 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:478 3.17 3:efb48653:::rbd_data.11826496459e.0000000000000007:3 [sparse-read 0~52418] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.160604+0000", 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.089488298999999993, 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000654953, 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 478 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160604+0000", 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160604+0000", 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160605+0000", 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7499999999999998e-07 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160609+0000", 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4320000000000001e-06 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160624+0000", 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-05 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160632+0000", 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4700000000000002e-06 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160686+0000", 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3810999999999997e-05 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.160737+0000", 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0563999999999999e-05 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.161259+0000", 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000521821 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:483 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.165900+0000", 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.084192059999999999, 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0024730400000000001, 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.343 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 483 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.165900+0000", 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.165900+0000", 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.165901+0000", 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1739999999999999e-06 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.165936+0000", 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4755000000000002e-05 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.165948+0000", 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1853e-05 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.165954+0000", 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4010000000000002e-06 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.166009+0000", 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5342e-05 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.166148+0000", 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013818200000000001 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.166304+0000", 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015642299999999999 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.167130+0000", 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00082578899999999995 2024-05-08T03:19:26.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168270+0000", 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011405180000000001 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168330+0000", 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9558000000000001e-05 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168373+0000", 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3044999999999999e-05 2024-05-08T03:19:26.348 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:484 3.12 3:4fe66973:::rbd_header.1182b3dbc99f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.168489+0000", 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.081603245000000005, 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001189336, 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:26.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 484 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168489+0000", 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168489+0000", 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168490+0000", 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.23e-07 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168494+0000", 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3200000000000001e-06 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168504+0000", 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0149999999999999e-05 2024-05-08T03:19:26.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168509+0000", 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4439999999999999e-06 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168554+0000", 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4546999999999999e-05 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168635+0000", 2024-05-08T03:19:26.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1228000000000002e-05 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.168863+0000", 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000227815 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.169313+0000", 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00045014000000000002 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.169623+0000", 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030941200000000001 2024-05-08T03:19:26.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.169653+0000", 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0552000000000002e-05 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.169678+0000", 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4905e-05 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.355 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:26.356 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:26.357 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:26.357 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:26.357 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:26.357 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:26.358 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:26.358 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:26.358 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:26.354+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:26.369 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:26.369 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:26.386 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:26.387 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:26.457 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:26.455+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:26.475 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:26.475 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:26.475 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:26.475 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:26.484 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:26.557 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:26.556+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:26.588 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:26.588 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:26.588 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:26.588 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:26.588 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:26.597 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:26.658 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:26.656+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:20 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.757281+0000", 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.983413629999999, 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0065633569999999997, 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.743 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757281+0000", 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757281+0000", 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757283+0000", 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4330000000000001e-06 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757539+0000", 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025649100000000002 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757562+0000", 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2965000000000001e-05 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757575+0000", 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3179999999999999e-05 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757644+0000", 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8690999999999997e-05 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.757668+0000", 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3977000000000001e-05 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763783+0000", 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0061144149999999998 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763845+0000", 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2205000000000005e-05 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:21 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.758060+0000", 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.982634997, 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0058140400000000004, 2024-05-08T03:19:26.747 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758060+0000", 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758060+0000", 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758061+0000", 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8999999999999996e-07 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758120+0000", 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9466000000000003e-05 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758130+0000", 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9939999999999995e-06 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758138+0000", 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.011e-06 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758188+0000", 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0312000000000002e-05 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.758208+0000", 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9794000000000001e-05 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763849+0000", 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0056411029999999997 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.763874+0000", 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4769999999999998e-05 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:30 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'16, mlcod=17'14)", 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.779003+0000", 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.96169188, 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0057940429999999996, 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.752 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779003+0000", 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779003+0000", 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779005+0000", 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.68e-06 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779018+0000", 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3192e-05 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779040+0000", 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2092000000000001e-05 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779052+0000", 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2014999999999999e-05 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779131+0000", 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9332000000000005e-05 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779156+0000", 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4567e-05 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.784676+0000", 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0055198449999999998 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.784797+0000", 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012132 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:31 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'17, mlcod=17'14)", 2024-05-08T03:19:26.756 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.961251325999999, 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063259120000000004, 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779444+0000", 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7899999999999999e-07 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779450+0000", 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7490000000000004e-06 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779459+0000", 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4029999999999996e-06 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779466+0000", 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4160000000000001e-06 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779512+0000", 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5731e-05 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779529+0000", 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7220000000000001e-05 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.784801+0000", 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0052718189999999996 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.785770+0000", 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000969095 2024-05-08T03:19:26.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:32 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.779809+0000", 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.960885604, 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063913019999999997, 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779809+0000", 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779809+0000", 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779810+0000", 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6599999999999996e-07 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779866+0000", 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6156000000000002e-05 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779878+0000", 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.183e-05 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779885+0000", 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8730000000000001e-06 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779940+0000", 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5047000000000003e-05 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.779956+0000", 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6427999999999999e-05 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.785772+0000", 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0058157809999999999 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786201+0000", 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042852100000000001 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:33 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'19, mlcod=17'14)", 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.780398+0000", 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.960296672, 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.005828031, 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.765 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780398+0000", 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780398+0000", 2024-05-08T03:19:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780399+0000", 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0699999999999997e-07 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780406+0000", 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8000000000000001e-06 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780414+0000", 2024-05-08T03:19:26.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1149999999999994e-06 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780420+0000", 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7529999999999998e-06 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780463+0000", 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3742999999999998e-05 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780479+0000", 2024-05-08T03:19:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6116e-05 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786203+0000", 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.005723672 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786226+0000", 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3325000000000001e-05 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:34 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'20, mlcod=17'14)", 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.780792+0000", 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.959902680999999, 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0054587849999999999, 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780792+0000", 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780792+0000", 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780793+0000", 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0100000000000005e-07 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780798+0000", 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9740000000000001e-06 2024-05-08T03:19:26.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780807+0000", 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8340000000000005e-06 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780813+0000", 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0680000000000002e-06 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780869+0000", 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6026000000000002e-05 2024-05-08T03:19:26.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.780886+0000", 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7130999999999999e-05 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786228+0000", 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0053418759999999997 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.786251+0000", 2024-05-08T03:19:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3374999999999999e-05 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:37 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'14)", 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.958521307, 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0056224350000000003, 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782174+0000", 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7999999999999995e-07 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782179+0000", 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8690000000000003e-06 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782189+0000", 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3850000000000008e-06 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782195+0000", 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782253+0000", 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8758000000000002e-05 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.782270+0000", 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6784000000000001e-05 2024-05-08T03:19:26.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.787742+0000", 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0054716820000000003 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.787796+0000", 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4423999999999999e-05 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.778 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:69 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'39)", 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.898175288000001, 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0061802039999999999, 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842520+0000", 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5000000000000002e-07 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842554+0000", 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3872999999999999e-05 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842566+0000", 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2089999999999999e-05 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842573+0000", 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5069999999999997e-06 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842640+0000", 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6698000000000004e-05 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.842661+0000", 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1345e-05 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848635+0000", 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0059744990000000003 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848700+0000", 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4542000000000006e-05 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.782 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:70 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'39)", 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.897448247, 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0054836540000000001, 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843247+0000", 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9299999999999998e-07 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843253+0000", 2024-05-08T03:19:26.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3890000000000004e-06 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843264+0000", 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1712000000000001e-05 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843271+0000", 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3779999999999998e-06 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843339+0000", 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7753e-05 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.843363+0000", 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4422000000000001e-05 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848703+0000", 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0053395889999999996 2024-05-08T03:19:26.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.848730+0000", 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7818e-05 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:72 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'46, mlcod=17'43)", 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.850881+0000", 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.889813808, 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063445560000000003, 2024-05-08T03:19:26.787 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850881+0000", 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850881+0000", 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850882+0000", 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.015e-06 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.850985+0000", 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010293099999999999 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851008+0000", 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2881999999999999e-05 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851016+0000", 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4340000000000003e-06 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851178+0000", 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016234100000000001 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851203+0000", 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4779000000000001e-05 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857164+0000", 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.005961002 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857226+0000", 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2172e-05 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.791 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:73 2.0 e17/16 2:30306672:devicehealth::main.db.0000000000000000:head v 17'47, mlcod=17'43)", 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.889662028, 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062212869999999998, 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851033+0000", 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851095+0000", 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1148999999999993e-05 2024-05-08T03:19:26.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851106+0000", 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1336e-05 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851113+0000", 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0720000000000002e-06 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851600+0000", 2024-05-08T03:19:26.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00048737100000000001 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.851620+0000", 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9660000000000002e-05 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857229+0000", 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0056093189999999998 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:13.857254+0000", 2024-05-08T03:19:26.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4975999999999999e-05 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:105 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'64)", 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.662618427, 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.006530821, 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078077+0000", 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4499999999999997e-07 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078083+0000", 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078097+0000", 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4508999999999999e-05 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078104+0000", 2024-05-08T03:19:26.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3829999999999997e-06 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078183+0000", 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9265999999999996e-05 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078198+0000", 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.503e-05 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084567+0000", 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0063695669999999996 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084607+0000", 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9950999999999997e-05 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:106 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'64)", 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.078561+0000", 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.662134043, 2024-05-08T03:19:26.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0060741110000000001, 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078561+0000", 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078561+0000", 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078562+0000", 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3099999999999997e-07 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078613+0000", 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1532000000000001e-05 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078627+0000", 2024-05-08T03:19:26.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4116e-05 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078634+0000", 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2310000000000001e-06 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078691+0000", 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7297999999999999e-05 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.078708+0000", 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6920999999999999e-05 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084610+0000", 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0059019540000000001 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084635+0000", 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5428000000000001e-05 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.804 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:108 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'70, mlcod=17'64)", 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.079739+0000", 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.660955803, 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0059717759999999998, 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079739+0000", 2024-05-08T03:19:26.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079739+0000", 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079740+0000", 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5700000000000002e-07 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079745+0000", 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5049999999999998e-06 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079758+0000", 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2697000000000001e-05 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.079764+0000", 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3740000000000003e-06 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.080136+0000", 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00037181099999999999 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.080156+0000", 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0244000000000001e-05 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085668+0000", 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0055119979999999997 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085711+0000", 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2589999999999997e-05 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:113 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'75, mlcod=17'65)", 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.082974+0000", 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657720792999999, 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062840980000000001, 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.809 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082974+0000", 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082974+0000", 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082975+0000", 2024-05-08T03:19:26.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6599999999999996e-07 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082983+0000", 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1300000000000001e-06 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.082993+0000", 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0128e-05 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083000+0000", 2024-05-08T03:19:26.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6529999999999997e-06 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083104+0000", 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010425 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083122+0000", 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8016000000000002e-05 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089197+0000", 2024-05-08T03:19:26.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0060752660000000002 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089258+0000", 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0989000000000002e-05 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:114 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'76, mlcod=17'65)", 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657663063999999, 2024-05-08T03:19:26.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.006255783, 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083032+0000", 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0800000000000001e-07 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083037+0000", 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5820000000000003e-06 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083044+0000", 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5070000000000003e-06 2024-05-08T03:19:26.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083049+0000", 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8269999999999997e-06 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084314+0000", 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0012644589999999999 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084333+0000", 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9287e-05 2024-05-08T03:19:26.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089262+0000", 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0049288439999999999 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.089288+0000", 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5969000000000002e-05 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:115 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'65)", 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656748552, 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014688414, 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083947+0000", 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1900000000000001e-07 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083985+0000", 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7769999999999999e-05 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084001+0000", 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.592e-05 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084009+0000", 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9109999999999993e-06 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084686+0000", 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00067690100000000004 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084703+0000", 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.755e-05 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098569+0000", 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013865446999999999 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098635+0000", 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6095999999999999e-05 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.821 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:116 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'65)", 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656555683000001, 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014525575000000001, 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084140+0000", 2024-05-08T03:19:26.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1900000000000003e-07 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084144+0000", 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6149999999999999e-06 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084153+0000", 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6459999999999999e-06 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084159+0000", 2024-05-08T03:19:26.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0079999999999999e-06 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085434+0000", 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001275263 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085455+0000", 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0893000000000001e-05 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098638+0000", 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013182523 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098665+0000", 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7107999999999999e-05 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:117 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'65)", 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:26.826 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.655887635999999, 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013907645, 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:26.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084808+0000", 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-07 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084813+0000", 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2259999999999996e-06 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084822+0000", 2024-05-08T03:19:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5830000000000007e-06 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084830+0000", 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9650000000000008e-06 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085897+0000", 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001067496 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085917+0000", 2024-05-08T03:19:26.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0080000000000001e-05 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098667+0000", 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012749462 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098715+0000", 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8285999999999998e-05 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:26.830 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:26.831 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:26.832 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:26.830+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:26.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:26.853 INFO:teuthology.orchestra.run.smithi177.stdout:flattening image image_client.0-clone1 2024-05-08T03:19:26.918 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-05-08T03:19:26.933 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:26.931+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:26.946 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:26.947 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:26.947 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:26.947 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:26.959 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:27.034 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:27.032+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:27.110 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:27.111 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:27.111 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:27.111 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:27.111 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:27.120 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:27.144 INFO:teuthology.orchestra.run.smithi177.stdout:162 clone 3 order 18 su 4096 sc 2 2024-05-08T03:19:27.144 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:27.142+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:435 3.4 e23/19 3:22315cfb:::rbd_data.1182b3dbc99f.0000000000000019:head v 23'2, mlcod=0'0)", 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.109764+0000", 2024-05-08T03:19:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.116427469, 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00062666200000000005, 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109764+0000", 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109764+0000", 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109765+0000", 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.144e-06 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109776+0000", 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0288999999999999e-05 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109792+0000", 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6608000000000001e-05 2024-05-08T03:19:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109801+0000", 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9320000000000002e-06 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109873+0000", 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1896000000000004e-05 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109897+0000", 2024-05-08T03:19:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.374e-05 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110346+0000", 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00044950900000000001 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110391+0000", 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4543999999999999e-05 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:472 3.0 e23/19 3:02cf0f5e:::rbd_data.1182b3dbc99f.000000000000000c:head v 23'2, mlcod=0'0)", 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.156828+0000", 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.0693634059999999, 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001432689, 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156828+0000", 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156828+0000", 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156829+0000", 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2449999999999999e-06 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156843+0000", 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3397999999999999e-05 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156863+0000", 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0111000000000001e-05 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156874+0000", 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0689e-05 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156937+0000", 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3864999999999999e-05 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156962+0000", 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.499e-05 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.158216+0000", 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0012537959999999999 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.158261+0000", 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4595000000000002e-05 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:544 3.0 e24/19 3:04479396:::rbd_id.image_client.0-clone2:head v 24'3, mlcod=23'1)", 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.39633662800000002, 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00072031500000000002, 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.68e-07 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829860+0000", 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829872+0000", 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2109999999999999e-05 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829880+0000", 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9840000000000001e-06 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829939+0000", 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9005999999999997e-05 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829963+0000", 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4530999999999999e-05 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.830533+0000", 2024-05-08T03:19:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00056923499999999997 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.830575+0000", 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2734000000000001e-05 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.240 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:27.242 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:27.242 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:27.240+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:27.342 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:27.341+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:27.388 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:27.389 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:27.389 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:27.389 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:27.397 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:27.482 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:27.480+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:27.542 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:27.542 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:27.542 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:27.542 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:27.543 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:27.544 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:27.542+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:27.551 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:27.644 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:27.643+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:27.696 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:497 3.2 3:438806ba:::rbd_object_map.1182b3dbc99f:head [read 0~37] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.799868+0000", 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.89496084600000003, 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00019913900000000001, 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:27.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 497 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799868+0000", 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799868+0000", 2024-05-08T03:19:27.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799870+0000", 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2500000000000001e-06 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799875+0000", 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3839999999999996e-06 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799887+0000", 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2265e-05 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799896+0000", 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1449999999999997e-06 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799943+0000", 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6292999999999999e-05 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.799994+0000", 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1252999999999999e-05 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.800067+0000", 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3548999999999997e-05 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:499 3.2 3:438806ba:::rbd_object_map.1182b3dbc99f:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.803006+0000", 2024-05-08T03:19:27.701 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.89182359600000005, 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0013766220000000001, 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 499 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803006+0000", 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803006+0000", 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803006+0000", 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7800000000000001e-07 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803010+0000", 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9020000000000001e-06 2024-05-08T03:19:27.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803020+0000", 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5459999999999997e-06 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803026+0000", 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5699999999999998e-06 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803068+0000", 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.163e-05 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803129+0000", 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1577999999999999e-05 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803281+0000", 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015195200000000001 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.803881+0000", 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059980099999999996 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804330+0000", 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000449071 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804360+0000", 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9785999999999999e-05 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804382+0000", 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2107999999999999e-05 2024-05-08T03:19:27.706 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop_reply(client.4482.0:499 3.2 e23/19 ondisk, result = 0)", 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.804273+0000", 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.89055637600000004, 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000113422, 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804273+0000", 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804273+0000", 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804273+0000", 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5499999999999998e-07 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804276+0000", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5239999999999999e-06 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804283+0000", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6149999999999999e-06 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804291+0000", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8089999999999993e-06 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804321+0000", 2024-05-08T03:19:27.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9374e-05 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804327+0000", 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7070000000000003e-06 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.804386+0000", 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8838000000000001e-05 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.710 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:588 3.2 3:438806ba:::rbd_object_map.1182b3dbc99f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.892037+0000", 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.802791861, 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0012125250000000001, 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 588 2024-05-08T03:19:27.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892037+0000", 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892037+0000", 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892039+0000", 2024-05-08T03:19:27.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0360000000000001e-06 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892050+0000", 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0312999999999999e-05 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892071+0000", 2024-05-08T03:19:27.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1246e-05 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892081+0000", 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0312000000000001e-05 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892151+0000", 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9779999999999999e-05 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892223+0000", 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1965999999999995e-05 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892346+0000", 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012265600000000001 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.892856+0000", 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051015900000000005 2024-05-08T03:19:27.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893187+0000", 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033135899999999998 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893219+0000", 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2199999999999997e-05 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893250+0000", 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0497999999999998e-05 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop_reply(client.4482.0:588 3.2 e24/19 ondisk, result = 0)", 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.893128+0000", 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.80170104399999997, 2024-05-08T03:19:27.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012735700000000001, 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893128+0000", 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893128+0000", 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893129+0000", 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7199999999999999e-07 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893132+0000", 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8770000000000002e-06 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893139+0000", 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7160000000000003e-06 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893147+0000", 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8890000000000002e-06 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893179+0000", 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1853000000000001e-05 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893184+0000", 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8729999999999998e-06 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.893255+0000", 2024-05-08T03:19:27.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1576999999999996e-05 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:633 3.8 3:124ead39:::rbd_data.11826496459e.0000000000000001:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966010+0000", 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72881872999999997, 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025153399999999998, 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 633 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966010+0000", 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966010+0000", 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966012+0000", 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9740000000000001e-06 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966023+0000", 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-05 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966047+0000", 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3991000000000001e-05 2024-05-08T03:19:27.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966059+0000", 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2074e-05 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966142+0000", 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3327999999999999e-05 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966262+0000", 2024-05-08T03:19:27.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119877 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:641 3.b 3:d66d02e4:::rbd_data.11826496459e.0000000000000009:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966273+0000", 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72855642200000004, 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027556500000000002, 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 641 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966273+0000", 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966273+0000", 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966273+0000", 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7999999999999995e-07 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966279+0000", 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.383e-06 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966287+0000", 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8359999999999992e-06 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966293+0000", 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3099999999999997e-06 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966442+0000", 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014875000000000001 2024-05-08T03:19:27.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966548+0000", 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000106706 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:644 3.18 3:1def0ac1:::rbd_data.11826496459e.000000000000000c:head [sparse-read 0~17426] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966310+0000", 2024-05-08T03:19:27.730 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72851887000000004, 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00028885299999999999, 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 644 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966310+0000", 2024-05-08T03:19:27.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966310+0000", 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966311+0000", 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6699999999999999e-07 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966313+0000", 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7319999999999998e-06 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966321+0000", 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0829999999999998e-06 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966326+0000", 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2560000000000002e-06 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966376+0000", 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9858999999999998e-05 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966447+0000", 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0982000000000006e-05 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966599+0000", 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015247399999999999 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:647 3.8 3:1041708f:::rbd_data.11826496459e.000000000000000f:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966483+0000", 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72834654600000004, 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000231148, 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 647 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.735 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966483+0000", 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966483+0000", 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966483+0000", 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2900000000000003e-07 2024-05-08T03:19:27.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966487+0000", 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1130000000000003e-06 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966497+0000", 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2410000000000001e-06 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966503+0000", 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5050000000000004e-06 2024-05-08T03:19:27.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966623+0000", 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119861 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966714+0000", 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0798999999999996e-05 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:652 3.7 3:e67c755a:::rbd_data.11826496459e.0000000000000014:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966528+0000", 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72830118300000002, 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000150712, 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 652 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966528+0000", 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966528+0000", 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966528+0000", 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.06e-07 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966532+0000", 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3009999999999998e-06 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966539+0000", 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966545+0000", 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4419999999999997e-06 2024-05-08T03:19:27.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966595+0000", 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9833999999999998e-05 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966679+0000", 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3907999999999997e-05 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.742 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:656 3.b 3:d4dbc86e:::rbd_data.11826496459e.0000000000000018:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966561+0000", 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72826802400000001, 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000131755, 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 656 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966561+0000", 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966561+0000", 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966562+0000", 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2399999999999998e-07 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966564+0000", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966570+0000", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8529999999999997e-06 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966576+0000", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3380000000000004e-06 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966619+0000", 2024-05-08T03:19:27.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3482000000000001e-05 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966693+0000", 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3818000000000004e-05 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:661 3.7 3:e5c4a50e:::rbd_data.11826496459e.000000000000001d:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966597+0000", 2024-05-08T03:19:27.746 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72823174400000001, 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018467900000000001, 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 661 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966597+0000", 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966597+0000", 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966598+0000", 2024-05-08T03:19:27.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966601+0000", 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.117e-06 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966607+0000", 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9850000000000003e-06 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966612+0000", 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8740000000000003e-06 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966701+0000", 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9188000000000001e-05 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966782+0000", 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1019000000000005e-05 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:666 3.b 3:d26f7a74:::rbd_data.11826496459e.0000000000000022:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966678+0000", 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72815084699999999, 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014314699999999999, 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.751 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 666 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966678+0000", 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966678+0000", 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966679+0000", 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.99e-07 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966682+0000", 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.365e-06 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966690+0000", 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9030000000000004e-06 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966697+0000", 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9090000000000003e-06 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966743+0000", 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6128000000000003e-05 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966821+0000", 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8243000000000003e-05 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:667 3.b 3:d6aa2b32:::rbd_data.11826496459e.0000000000000023:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966712+0000", 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.72811759099999995, 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000222337, 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.755 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 667 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966712+0000", 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966712+0000", 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966712+0000", 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8400000000000004e-07 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966715+0000", 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.509e-06 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966721+0000", 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8090000000000004e-06 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966727+0000", 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1039999999999998e-06 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966845+0000", 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000118612 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966934+0000", 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8719000000000002e-05 2024-05-08T03:19:27.759 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:672 3.7 3:e54e0f6f:::rbd_data.11826496459e.0000000000000028:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.966868+0000", 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.727961423, 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000145592, 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:27.760 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 672 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966868+0000", 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966868+0000", 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966869+0000", 2024-05-08T03:19:27.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4099999999999997e-07 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966873+0000", 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9580000000000001e-06 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966883+0000", 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0278999999999999e-05 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:27.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966889+0000", 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1929999999999998e-06 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.966934+0000", 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5334000000000003e-05 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.967013+0000", 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8986999999999994e-05 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.763 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:27.764 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:27.764 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:27.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:27.766 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:27.763+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:27.807 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:19:27.866 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:27.864+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:27.866 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone2 from 0x3c4871c (overlap 0x3c4871c) to 0x2a32b60 2024-05-08T03:19:27.894 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:27.894 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:27.894 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:27.894 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:27.904 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:27.956 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-05-08T03:19:27.988 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:27.986+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:28.026 INFO:teuthology.orchestra.run.smithi177.stdout:163 trunc from 0x3c4871c to 0x5ca5e0f 2024-05-08T03:19:28.038 INFO:teuthology.orchestra.run.smithi177.stdout:164 write 0x32a2e19 thru 0x32ab12e (0x8316 bytes) 2024-05-08T03:19:28.054 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.054 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.054 INFO:teuthology.orchestra.run.smithi177.stdout:171 read 0x2419c4 thru 0x250d40 (0xf37d bytes) 2024-05-08T03:19:28.059 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:28.059 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:28.059 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:28.059 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:28.059 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:28.065 INFO:teuthology.orchestra.run.smithi177.stdout:172 write 0x8992f74 thru 0x899b444 (0x84d1 bytes) 2024-05-08T03:19:28.066 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:28.065+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:28.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:28.078 INFO:teuthology.orchestra.run.smithi177.stdout:174 trunc from 0x899b445 to 0xcb431b2 2024-05-08T03:19:28.083 INFO:teuthology.orchestra.run.smithi177.stdout:175 read 0xa3346a4 thru 0xa33da7f (0x93dc bytes) 2024-05-08T03:19:28.083 INFO:teuthology.orchestra.run.smithi177.stdout:176 punch from 0x48cd249 to 0x48da0eb, (0xcea2 bytes) 2024-05-08T03:19:28.084 INFO:teuthology.orchestra.run.smithi177.stdout:178 read 0x3072cf6 thru 0x307d926 (0xac31 bytes) 2024-05-08T03:19:28.086 INFO:teuthology.orchestra.run.smithi177.stdout:179 write 0x3395b28 thru 0x339b030 (0x5509 bytes) 2024-05-08T03:19:28.106 INFO:teuthology.orchestra.run.smithi177.stdout:180 write 0x793b1af thru 0x794709d (0xbeef bytes) 2024-05-08T03:19:28.112 INFO:teuthology.orchestra.run.smithi177.stdout:181 punch from 0xc10ce91 to 0xc1135e7, (0x6756 bytes) 2024-05-08T03:19:28.112 INFO:teuthology.orchestra.run.smithi177.stdout:182 read 0x8daa927 thru 0x8dad502 (0x2bdc bytes) 2024-05-08T03:19:28.112 INFO:teuthology.orchestra.run.smithi177.stdout:183 read 0x1e574c5 thru 0x1e60bcc (0x9708 bytes) 2024-05-08T03:19:28.112 INFO:teuthology.orchestra.run.smithi177.stdout:184 write 0x6ef86f0 thru 0x6f069f6 (0xe307 bytes) 2024-05-08T03:19:28.120 INFO:teuthology.orchestra.run.smithi177.stdout:187 write 0x80cd32e thru 0x80dcce1 (0xf9b4 bytes) 2024-05-08T03:19:28.125 INFO:teuthology.orchestra.run.smithi177.stdout:188 write 0x8abc26a thru 0x8ac14e3 (0x527a bytes) 2024-05-08T03:19:28.130 INFO:teuthology.orchestra.run.smithi177.stdout:190 write 0x3c47111 thru 0x3c559ab (0xe89b bytes) 2024-05-08T03:19:28.144 INFO:teuthology.orchestra.run.smithi177.stdout:191 read 0x7968722 thru 0x7974129 (0xba08 bytes) 2024-05-08T03:19:28.145 INFO:teuthology.orchestra.run.smithi177.stdout:192 write 0x85fc1a0 thru 0x860a78a (0xe5eb bytes) 2024-05-08T03:19:28.156 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.156 INFO:teuthology.orchestra.run.smithi177.stdout:195 read 0x24b5ee9 thru 0x24b70d6 (0x11ee bytes) 2024-05-08T03:19:28.156 INFO:teuthology.orchestra.run.smithi177.stdout:198 punch from 0xa0a2cc2 to 0xa0a7a07, (0x4d45 bytes) 2024-05-08T03:19:28.156 INFO:teuthology.orchestra.run.smithi177.stdout:199 write 0x160cf1a thru 0x1618b5e (0xbc45 bytes) 2024-05-08T03:19:28.167 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:28.166+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:28.181 INFO:teuthology.orchestra.run.smithi177.stdout:200 punch from 0x356c557 to 0x3570686, (0x412f bytes) 2024-05-08T03:19:28.187 INFO:teuthology.orchestra.run.smithi177.stdout:204 write 0xd9d61f0 thru 0xd9e146b (0xb27c bytes) 2024-05-08T03:19:28.194 INFO:teuthology.orchestra.run.smithi177.stdout:205 writesame 0x7d20edb thru 0x7d21ea1 data_size 0x1(0xfc7 bytes) 2024-05-08T03:19:28.198 INFO:teuthology.orchestra.run.smithi177.stdout:207 read 0x9783afd thru 0x9792356 (0xe85a bytes) 2024-05-08T03:19:28.198 INFO:teuthology.orchestra.run.smithi177.stdout:208 writesame 0x1ec4b3b thru 0x1ed05dd data_size 0x1(0xbaa3 bytes) 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.36070280500000002, 2024-05-08T03:19:28.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.232 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:19:28.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:19:28.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.36036062699999999, 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:19:28.238 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.243 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.360321795, 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.244 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:28.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:28.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:19:28.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:19:28.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.360275762, 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:19:28.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.35925857, 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:19:28.257 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:28.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:19:28.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:19:28.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:19:28.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.35922231900000001, 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.263 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:19:28.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:19:28.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:19:28.268 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.35919501999999998, 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.269 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.35916532699999998, 2024-05-08T03:19:28.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:28.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:19:28.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.281 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.359139812, 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:19:28.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.287 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.34902564600000002, 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.288 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.293 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096686801000000003, 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:28.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:19:28.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:19:28.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.298 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096668470000000006, 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.302 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096644009000000003, 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.303 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:19:28.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:19:28.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.307 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096617666000000005, 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.308 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:19:28.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096600274999999999, 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:19:28.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:19:28.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096582994000000005, 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.317 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:19:28.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096555557, 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.322 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:19:28.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:19:28.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096537598000000002, 2024-05-08T03:19:28.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096520512000000003, 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.331 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:28.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:28.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:19:28.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:19:28.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.096503528000000005, 2024-05-08T03:19:28.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:19:28.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:28.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.341 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:28.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:28.344 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.344 INFO:teuthology.orchestra.run.smithi177.stdout:211 write 0xbced3c0 thru 0xbcf906f (0xbcb0 bytes) 2024-05-08T03:19:28.344 INFO:teuthology.orchestra.run.smithi177.stdout:212 punch from 0x5385bec to 0x5394683, (0xea97 bytes) 2024-05-08T03:19:28.344 INFO:teuthology.orchestra.run.smithi177.stdout:213 writesame 0x3e88303 thru 0x3e8fa09 data_size 0x7(0x7707 bytes) 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:214 read 0x5fb04d6 thru 0x5fb6ec5 (0x69f0 bytes) 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:215 read 0x595befa thru 0x5969fc9 (0xe0d0 bytes) 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:216 trunc from 0xd9e146c to 0xc66dba5 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:217 trunc from 0xc66dba5 to 0xe49fd4d 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:218 punch from 0x88e4e11 to 0x88ea0a0, (0x528f bytes) 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:219 trunc from 0xe49fd4d to 0x3f4f4a6 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:220 write 0xd2c97e4 thru 0xd2d634e (0xcb6b bytes) 2024-05-08T03:19:28.345 INFO:teuthology.orchestra.run.smithi177.stdout:221 trunc from 0xd2d634f to 0x410b621 2024-05-08T03:19:28.345 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:28.341+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:28.357 INFO:teuthology.orchestra.run.smithi177.stdout:222 read 0x14f5f1e thru 0x14fe0b4 (0x8197 bytes) 2024-05-08T03:19:28.359 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.376 INFO:teuthology.orchestra.run.smithi177.stdout:224 write 0xaf2818e thru 0xaf2efc8 (0x6e3b bytes) 2024-05-08T03:19:28.382 INFO:teuthology.orchestra.run.smithi177.stdout:226 writesame 0x1ad3b9d thru 0x1adefb8 data_size 0x2(0xb41c bytes) 2024-05-08T03:19:28.398 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:28.417 INFO:teuthology.orchestra.run.smithi177.stdout:229 trunc from 0xaf2efc9 to 0x8f6152 2024-05-08T03:19:28.443 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:28.441+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:28.459 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.481 INFO:teuthology.orchestra.run.smithi177.stdout:231 write 0xde4e1a2 thru 0xde50a01 (0x2860 bytes) 2024-05-08T03:19:28.485 INFO:teuthology.orchestra.run.smithi177.stdout:232 read 0xb876e21 thru 0xb8865ed (0xf7cd bytes) 2024-05-08T03:19:28.485 INFO:teuthology.orchestra.run.smithi177.stdout:233 write 0xb5633f4 thru 0xb56b740 (0x834d bytes) 2024-05-08T03:19:28.490 INFO:teuthology.orchestra.run.smithi177.stdout:234 read 0x1cbeae8 thru 0x1cc5471 (0x698a bytes) 2024-05-08T03:19:28.490 INFO:teuthology.orchestra.run.smithi177.stdout:235 write 0x52d1845 thru 0x52d53cc (0x3b88 bytes) 2024-05-08T03:19:28.494 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.494 INFO:teuthology.orchestra.run.smithi177.stdout:237 write 0xd5a6897 thru 0xd5b0041 (0x97ab bytes) 2024-05-08T03:19:28.497 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:28.497 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:28.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.497 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:2075 3.d e25/19 3:b4d8e50a:::rbd_data.1182c698972d.0000000000000356:head v 25'6, mlcod=25'4)", 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.496297+0000", 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.00018744500000000001, 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021025899999999999, 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:28.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496297+0000", 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496297+0000", 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496298+0000", 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6700000000000002e-07 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:28.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496313+0000", 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5299999999999999e-05 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496326+0000", 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2695e-05 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496333+0000", 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3259999999999998e-06 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496391+0000", 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7843000000000002e-05 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.496420+0000", 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9416000000000001e-05 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: ], 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 1 2024-05-08T03:19:28.501 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:28.502 INFO:teuthology.orchestra.run.smithi177.stdout:239 punch from 0x3c13cca to 0x3c18ef6, (0x522c bytes) 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:241 read 0xcca90f6 thru 0xccb337b (0xa286 bytes) 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:242 punch from 0x900a10d to 0x900cecb, (0x2dbe bytes) 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:246 read 0x779f97b thru 0x77aa250 (0xa8d6 bytes) 2024-05-08T03:19:28.503 INFO:teuthology.orchestra.run.smithi177.stdout:249 writesame 0xc414be5 thru 0xc41dbf1 data_size 0x1(0x900d bytes) 2024-05-08T03:19:28.507 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:28.514 INFO:teuthology.orchestra.run.smithi177.stdout:250 write 0xff3e5d thru 0x10003f4 (0xc598 bytes) 2024-05-08T03:19:28.520 INFO:teuthology.orchestra.run.smithi177.stdout:251 write 0x1b1e944 thru 0x1b2c82c (0xdee9 bytes) 2024-05-08T03:19:28.524 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.524 INFO:teuthology.orchestra.run.smithi177.stdout:253 read 0xc453849 thru 0xc460cc1 (0xd479 bytes) 2024-05-08T03:19:28.525 INFO:teuthology.orchestra.run.smithi177.stdout:255 writesame 0x6de0e58 thru 0x6de46f6 data_size 0xd(0x389f bytes) 2024-05-08T03:19:28.529 INFO:teuthology.orchestra.run.smithi177.stdout:258 read 0x2d765a4 thru 0x2d7e49e (0x7efb bytes) 2024-05-08T03:19:28.529 INFO:teuthology.orchestra.run.smithi177.stdout:259 writesame 0x3202e3f thru 0x320678d data_size 0x1(0x394f bytes) 2024-05-08T03:19:28.544 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:28.544 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:28.548 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:28.546+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:28.554 INFO:teuthology.orchestra.run.smithi177.stdout:260 trunc from 0xde50a02 to 0x1f5f7ef 2024-05-08T03:19:28.570 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:28.570 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:28.583 INFO:teuthology.orchestra.run.smithi177.stdout:261 writesame 0x6eb5769 thru 0x6ec234e data_size 0x1(0xcbe6 bytes) 2024-05-08T03:19:28.583 INFO:teuthology.orchestra.run.smithi177.stdout:262 writesame 0x25420bc thru 0x2548672 data_size 0x1(0x65b7 bytes) 2024-05-08T03:19:28.588 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:28.588 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:28.594 INFO:teuthology.orchestra.run.smithi177.stdout:263 writesame 0x5a6b807 thru 0x5a77b89 data_size 0x1(0xc383 bytes) 2024-05-08T03:19:28.607 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:28.607 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:28.611 INFO:teuthology.orchestra.run.smithi177.stdout:265 read 0x4c6ebed thru 0x4c79ccc (0xb0e0 bytes) 2024-05-08T03:19:28.611 INFO:teuthology.orchestra.run.smithi177.stdout:267 read 0x3f2c0b0 thru 0x3f384fe (0xc44f bytes) 2024-05-08T03:19:28.612 INFO:teuthology.orchestra.run.smithi177.stdout:269 read 0x20de49d thru 0x20e7b5e (0x96c2 bytes) 2024-05-08T03:19:28.617 INFO:teuthology.orchestra.run.smithi177.stdout:274 trunc from 0x6ec234f to 0xbb9c475 2024-05-08T03:19:28.626 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:28.626 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:28.628 INFO:teuthology.orchestra.run.smithi177.stdout:275 trunc from 0xbb9c475 to 0x72a8c8f 2024-05-08T03:19:28.635 INFO:teuthology.orchestra.run.smithi177.stdout:276 read 0x49138d5 thru 0x4918adf (0x520b bytes) 2024-05-08T03:19:28.635 INFO:teuthology.orchestra.run.smithi177.stdout:278 writesame 0x3372075 thru 0x337582b data_size 0x1(0x37b7 bytes) 2024-05-08T03:19:28.641 INFO:teuthology.orchestra.run.smithi177.stdout:279 punch from 0x316895e to 0x316d7bd, (0x4e5f bytes) 2024-05-08T03:19:28.645 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:28.643+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:28.646 INFO:teuthology.orchestra.run.smithi177.stdout:281 trunc from 0x72a8c8f to 0x3995694 2024-05-08T03:19:28.646 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:28.646 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:28.661 INFO:teuthology.orchestra.run.smithi177.stdout:284 writesame 0x3ebe516 thru 0x3ec607b data_size 0x51(0x7b66 bytes) 2024-05-08T03:19:28.661 INFO:teuthology.orchestra.run.smithi177.stdout:285 writesame 0xda1d79 thru 0xda6208 data_size 0x8(0x4490 bytes) 2024-05-08T03:19:28.665 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:28.665 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:28.668 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:28.668 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:28.677 INFO:teuthology.orchestra.run.smithi177.stdout:286 trunc from 0x3ec607c to 0xb89df8c 2024-05-08T03:19:28.678 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:28.682 INFO:teuthology.orchestra.run.smithi177.stdout:288 read 0x50f4f6b thru 0x50fbcc6 (0x6d5c bytes) 2024-05-08T03:19:28.683 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.683 INFO:teuthology.orchestra.run.smithi177.stdout:290 punch from 0x8acac0 to 0x8b3822, (0x6d62 bytes) 2024-05-08T03:19:28.683 INFO:teuthology.orchestra.run.smithi177.stdout:294 write 0x1e28bb6 thru 0x1e34df6 (0xc241 bytes) 2024-05-08T03:19:28.684 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:28.684 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:295 read 0x7fccce7 thru 0x7fd6819 (0x9b33 bytes) 2024-05-08T03:19:28.693 INFO:teuthology.orchestra.run.smithi177.stdout:296 read 0x7be2c7 thru 0x7c8356 (0xa090 bytes) 2024-05-08T03:19:28.694 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.694 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.694 INFO:teuthology.orchestra.run.smithi177.stdout:300 read 0xac1d0b0 thru 0xac21932 (0x4883 bytes) 2024-05-08T03:19:28.702 INFO:teuthology.orchestra.run.smithi177.stdout:301 write 0xcb6e72d thru 0xcb76e74 (0x8748 bytes) 2024-05-08T03:19:28.711 INFO:teuthology.orchestra.run.smithi177.stdout:302 read 0xe3ed76 thru 0xe3fd93 (0x101e bytes) 2024-05-08T03:19:28.719 INFO:teuthology.orchestra.run.smithi177.stdout:303 write 0xd0ec9a0 thru 0xd0f68ae (0x9f0f bytes) 2024-05-08T03:19:28.735 INFO:teuthology.orchestra.run.smithi177.stdout:304 writesame 0xe1ddb08 thru 0xe1e9213 data_size 0x21(0xb70c bytes) 2024-05-08T03:19:28.735 INFO:teuthology.orchestra.run.smithi177.stdout:305 writesame 0xda87fd2 thru 0xda9518e data_size 0x1(0xd1bd bytes) 2024-05-08T03:19:28.752 INFO:teuthology.orchestra.run.smithi177.stdout:306 read 0x6bed9ab thru 0x6bf9789 (0xbddf bytes) 2024-05-08T03:19:28.753 INFO:teuthology.orchestra.run.smithi177.stdout:307 write 0xaeb519 thru 0xaf438d (0x8e75 bytes) 2024-05-08T03:19:28.756 INFO:teuthology.orchestra.run.smithi177.stdout:309 write 0x5cbf74b thru 0x5ccb7c9 (0xc07f bytes) 2024-05-08T03:19:28.761 INFO:teuthology.orchestra.run.smithi177.stdout:310 read 0x6795243 thru 0x67966c4 (0x1482 bytes) 2024-05-08T03:19:28.761 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.761 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.761 INFO:teuthology.orchestra.run.smithi177.stdout:313 read 0x80beaa0 thru 0x80c49ba (0x5f1b bytes) 2024-05-08T03:19:28.761 INFO:teuthology.orchestra.run.smithi177.stdout:314 punch from 0x165351c to 0x16595ae, (0x6092 bytes) 2024-05-08T03:19:28.761 INFO:teuthology.orchestra.run.smithi177.stdout:315 read 0xd5d1caa thru 0xd5ddb89 (0xbee0 bytes) 2024-05-08T03:19:28.763 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:28.761+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:28.775 INFO:teuthology.orchestra.run.smithi177.stdout:316 trunc from 0xe1e9214 to 0x1da88d3 2024-05-08T03:19:28.787 INFO:teuthology.orchestra.run.smithi177.stdout:317 read 0xf20b6d thru 0xf29f96 (0x942a bytes) 2024-05-08T03:19:28.794 INFO:teuthology.orchestra.run.smithi177.stdout:318 trunc from 0x1da88d3 to 0x76b49d5 2024-05-08T03:19:28.802 INFO:teuthology.orchestra.run.smithi177.stdout:320 trunc from 0x76b49d5 to 0x4b39dbe 2024-05-08T03:19:28.806 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:19:28.808 INFO:teuthology.orchestra.run.smithi177.stdout:321 read 0x3c915f7 thru 0x3c98885 (0x728f bytes) 2024-05-08T03:19:28.808 INFO:teuthology.orchestra.run.smithi177.stdout:322 read 0x22a0b81 thru 0x22ad936 (0xcdb6 bytes) 2024-05-08T03:19:28.814 INFO:teuthology.orchestra.run.smithi177.stdout:323 writesame 0x53c8e7c thru 0x53d8a6f data_size 0x81(0xfbf4 bytes) 2024-05-08T03:19:28.814 INFO:teuthology.orchestra.run.smithi177.stdout:324 writesame 0x51a5cb3 thru 0x51aa871 data_size 0x1(0x4bbf bytes) 2024-05-08T03:19:28.822 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.822 INFO:teuthology.orchestra.run.smithi177.stdout:328 punch from 0x1d12b7f to 0x1d15828, (0x2ca9 bytes) 2024-05-08T03:19:28.822 INFO:teuthology.orchestra.run.smithi177.stdout:330 read 0x1135ef9 thru 0x1138814 (0x291c bytes) 2024-05-08T03:19:28.823 INFO:teuthology.orchestra.run.smithi177.stdout:332 write 0x10d4d97 thru 0x10d7c4c (0x2eb6 bytes) 2024-05-08T03:19:28.833 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:28.833 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:28.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:28.843 INFO:teuthology.orchestra.run.smithi177.stdout:333 writesame 0xc59187b thru 0xc5a0ce1 data_size 0x25(0xf467 bytes) 2024-05-08T03:19:28.843 INFO:teuthology.orchestra.run.smithi177.stdout:334 read 0x155ef5a thru 0x1562274 (0x331b bytes) 2024-05-08T03:19:28.843 INFO:teuthology.orchestra.run.smithi177.stdout:335 writesame 0x302ed57 thru 0x303cb8b data_size 0x1f(0xde35 bytes) 2024-05-08T03:19:28.851 INFO:teuthology.orchestra.run.smithi177.stdout:336 read 0x171f239 thru 0x1729160 (0x9f28 bytes) 2024-05-08T03:19:28.851 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.851 INFO:teuthology.orchestra.run.smithi177.stdout:339 writesame 0x656ecf5 thru 0x6570247 data_size 0x35(0x1553 bytes) 2024-05-08T03:19:28.863 INFO:teuthology.orchestra.run.smithi177.stdout:340 trunc from 0xc5a0ce2 to 0x6afd886 2024-05-08T03:19:28.868 INFO:teuthology.orchestra.run.smithi177.stdout:341 write 0x68e2f9c thru 0x68e695c (0x39c1 bytes) 2024-05-08T03:19:28.872 INFO:teuthology.orchestra.run.smithi177.stdout:343 punch from 0x35819eb to 0x358316e, (0x1783 bytes) 2024-05-08T03:19:28.872 INFO:teuthology.orchestra.run.smithi177.stdout:344 read 0x3f2d7a9 thru 0x3f31325 (0x3b7d bytes) 2024-05-08T03:19:28.872 INFO:teuthology.orchestra.run.smithi177.stdout:345 read 0x197697e thru 0x197a5bb (0x3c3e bytes) 2024-05-08T03:19:28.872 INFO:teuthology.orchestra.run.smithi177.stdout:347 write 0x3de8117 thru 0x3df5106 (0xcff0 bytes) 2024-05-08T03:19:28.878 INFO:teuthology.orchestra.run.smithi177.stdout:348 read 0x48f8d6c thru 0x48f9d21 (0xfb6 bytes) 2024-05-08T03:19:28.884 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:28.882+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:28.884 INFO:teuthology.orchestra.run.smithi177.stdout:349 writesame 0x808e066 thru 0x80941c0 data_size 0x1(0x615b bytes) 2024-05-08T03:19:28.884 INFO:teuthology.orchestra.run.smithi177.stdout:353 write 0x144519d thru 0x144bcd9 (0x6b3d bytes) 2024-05-08T03:19:28.888 INFO:teuthology.orchestra.run.smithi177.stdout:355 read 0xbcb5b0 thru 0xbce5c2 (0x3013 bytes) 2024-05-08T03:19:28.888 INFO:teuthology.orchestra.run.smithi177.stdout:356 punch from 0x5f57225 to 0x5f62022, (0xadfd bytes) 2024-05-08T03:19:28.888 INFO:teuthology.orchestra.run.smithi177.stdout:360 write 0xe3aade thru 0xe3d16a (0x268d bytes) 2024-05-08T03:19:28.892 INFO:teuthology.orchestra.run.smithi177.stdout:361 read 0x13749b thru 0x1402e7 (0x8e4d bytes) 2024-05-08T03:19:28.894 INFO:teuthology.orchestra.run.smithi177.stdout:364 read 0x11ddfce thru 0x11de000 (0x33 bytes) 2024-05-08T03:19:28.894 INFO:teuthology.orchestra.run.smithi177.stdout:365 write 0x787ebd6 thru 0x787f2bd (0x6e8 bytes) 2024-05-08T03:19:28.900 INFO:teuthology.orchestra.run.smithi177.stdout:367 punch from 0x7c9b12b to 0x7ca8aea, (0xd9bf bytes) 2024-05-08T03:19:28.900 INFO:teuthology.orchestra.run.smithi177.stdout:368 writesame 0x2d9a5c3 thru 0x2d9d56a data_size 0x3d(0x2fa8 bytes) 2024-05-08T03:19:28.904 INFO:teuthology.orchestra.run.smithi177.stdout:370 read 0x41370e8 thru 0x4138de2 (0x1cfb bytes) 2024-05-08T03:19:28.904 INFO:teuthology.orchestra.run.smithi177.stdout:371 write 0x65835e2 thru 0x6586410 (0x2e2f bytes) 2024-05-08T03:19:28.915 INFO:teuthology.orchestra.run.smithi177.stdout:372 write 0x879a31e thru 0x87a587a (0xb55d bytes) 2024-05-08T03:19:28.924 INFO:teuthology.orchestra.run.smithi177.stdout:373 trunc from 0x87a587b to 0x651ca50 2024-05-08T03:19:28.937 INFO:teuthology.orchestra.run.smithi177.stdout:374 read 0x13aa506 thru 0x13b80a8 (0xdba3 bytes) 2024-05-08T03:19:28.938 INFO:teuthology.orchestra.run.smithi177.stdout:375 write 0x4365fad thru 0x436ada8 (0x4dfc bytes) 2024-05-08T03:19:28.941 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:28.941 INFO:teuthology.orchestra.run.smithi177.stdout:377 read 0x43ad1cc thru 0x43b48bb (0x76f0 bytes) 2024-05-08T03:19:28.941 INFO:teuthology.orchestra.run.smithi177.stdout:378 write 0x5719c5c thru 0x572900a (0xf3af bytes) 2024-05-08T03:19:28.946 INFO:teuthology.orchestra.run.smithi177.stdout:379 writesame 0x1b5cf74 thru 0x1b60434 data_size 0x25(0x34c1 bytes) 2024-05-08T03:19:28.948 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:28.946+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:28.958 INFO:teuthology.orchestra.run.smithi177.stdout:380 write 0x96075b5 thru 0x96100bf (0x8b0b bytes) 2024-05-08T03:19:28.965 INFO:teuthology.orchestra.run.smithi177.stdout:381 read 0x32bd0e1 thru 0x32c7109 (0xa029 bytes) 2024-05-08T03:19:28.970 INFO:teuthology.orchestra.run.smithi177.stdout:382 trunc from 0x96100c0 to 0xd806b74 2024-05-08T03:19:28.976 INFO:teuthology.orchestra.run.smithi177.stdout:383 write 0x8970a99 thru 0x89785d9 (0x7b41 bytes) 2024-05-08T03:19:28.980 INFO:teuthology.orchestra.run.smithi177.stdout:384 writesame 0xb38de02 thru 0xb39caf1 data_size 0x44(0xecf0 bytes) 2024-05-08T03:19:28.986 INFO:teuthology.orchestra.run.smithi177.stdout:386 trunc from 0xd806b74 to 0xdc97e69 2024-05-08T03:19:28.996 INFO:teuthology.orchestra.run.smithi177.stdout:387 trunc from 0xdc97e69 to 0x98e544f 2024-05-08T03:19:29.000 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.000 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.008 INFO:teuthology.orchestra.run.smithi177.stdout:388 writesame 0x192662d thru 0x193625f data_size 0x1(0xfc33 bytes) 2024-05-08T03:19:29.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:29.027 INFO:teuthology.orchestra.run.smithi177.stdout:390 trunc from 0x98e544f to 0x8f0aa21 2024-05-08T03:19:29.039 INFO:teuthology.orchestra.run.smithi177.stdout:391 write 0x8c2ea18 thru 0x8c30cc8 (0x22b1 bytes) 2024-05-08T03:19:29.043 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.043 INFO:teuthology.orchestra.run.smithi177.stdout:393 read 0x2647966 thru 0x264a1a6 (0x2841 bytes) 2024-05-08T03:19:29.044 INFO:teuthology.orchestra.run.smithi177.stdout:395 read 0x1724a67 thru 0x173060e (0xbba8 bytes) 2024-05-08T03:19:29.044 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.044 INFO:teuthology.orchestra.run.smithi177.stdout:398 write 0x601a03d thru 0x6020418 (0x63dc bytes) 2024-05-08T03:19:29.048 INFO:teuthology.orchestra.run.smithi177.stdout:399 write 0x78d6e14 thru 0x78de9e5 (0x7bd2 bytes) 2024-05-08T03:19:29.049 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:29.047+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:29.062 INFO:teuthology.orchestra.run.smithi177.stdout:401 write 0xb54f460 thru 0xb55a1c9 (0xad6a bytes) 2024-05-08T03:19:29.066 INFO:teuthology.orchestra.run.smithi177.stdout:403 punch from 0x4ffe02c to 0x500ba96, (0xda6a bytes) 2024-05-08T03:19:29.076 INFO:teuthology.orchestra.run.smithi177.stdout:406 write 0xc549f64 thru 0xc557384 (0xd421 bytes) 2024-05-08T03:19:29.086 INFO:teuthology.orchestra.run.smithi177.stdout:407 write 0xcf7053b thru 0xcf7c1ca (0xbc90 bytes) 2024-05-08T03:19:29.092 INFO:teuthology.orchestra.run.smithi177.stdout:408 write 0x677c435 thru 0x67895d0 (0xd19c bytes) 2024-05-08T03:19:29.101 INFO:teuthology.orchestra.run.smithi177.stdout:409 read 0x534da2f thru 0x534f25c (0x182e bytes) 2024-05-08T03:19:29.101 INFO:teuthology.orchestra.run.smithi177.stdout:410 writesame 0x77ae36a thru 0x77b4fb0 data_size 0x1(0x6c47 bytes) 2024-05-08T03:19:29.113 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.113 INFO:teuthology.orchestra.run.smithi177.stdout:412 punch from 0x243481e to 0x2438ca1, (0x4483 bytes) 2024-05-08T03:19:29.113 INFO:teuthology.orchestra.run.smithi177.stdout:413 read 0x7edd72d thru 0x7ee5adc (0x83b0 bytes) 2024-05-08T03:19:29.121 INFO:teuthology.orchestra.run.smithi177.stdout:415 writesame 0xe9a7e01 thru 0xe9b38d8 data_size 0xc(0xbad8 bytes) 2024-05-08T03:19:29.121 INFO:teuthology.orchestra.run.smithi177.stdout:417 write 0x8f0b554 thru 0x8f12103 (0x6bb0 bytes) 2024-05-08T03:19:29.126 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.130 INFO:teuthology.orchestra.run.smithi177.stdout:420 trunc from 0xe9b38d9 to 0xac47d60 2024-05-08T03:19:29.149 INFO:teuthology.orchestra.run.smithi177.stdout:421 read 0x995b8e8 thru 0x996939f (0xdab8 bytes) 2024-05-08T03:19:29.150 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:29.148+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:29.150 INFO:teuthology.orchestra.run.smithi177.stdout:422 trunc from 0xac47d60 to 0xbcbe06d 2024-05-08T03:19:29.156 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.156 INFO:teuthology.orchestra.run.smithi177.stdout:425 write 0xa09153e thru 0xa098e9a (0x795d bytes) 2024-05-08T03:19:29.162 INFO:teuthology.orchestra.run.smithi177.stdout:426 read 0x19af9e2 thru 0x19b8463 (0x8a82 bytes) 2024-05-08T03:19:29.162 INFO:teuthology.orchestra.run.smithi177.stdout:427 read 0x617a707 thru 0x617d9dc (0x32d6 bytes) 2024-05-08T03:19:29.163 INFO:teuthology.orchestra.run.smithi177.stdout:428 punch from 0x8cbb1e1 to 0x8cbeb71, (0x3990 bytes) 2024-05-08T03:19:29.167 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.167 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.170 INFO:teuthology.orchestra.run.smithi177.stdout:429 trunc from 0xbcbe06d to 0x5110441 2024-05-08T03:19:29.177 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:29.188 INFO:teuthology.orchestra.run.smithi177.stdout:434 trunc from 0x5110441 to 0x62d26d4 2024-05-08T03:19:29.210 INFO:teuthology.orchestra.run.smithi177.stdout:435 writesame 0xec1fe9b thru 0xec22752 data_size 0x4(0x28b8 bytes) 2024-05-08T03:19:29.211 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.211 INFO:teuthology.orchestra.run.smithi177.stdout:438 write 0x8c77a3d thru 0x8c77fc3 (0x587 bytes) 2024-05-08T03:19:29.214 INFO:teuthology.orchestra.run.smithi177.stdout:439 punch from 0x60005a8 to 0x60063cb, (0x5e23 bytes) 2024-05-08T03:19:29.227 INFO:teuthology.orchestra.run.smithi177.stdout:440 trunc from 0xec22753 to 0x41eea1a 2024-05-08T03:19:29.245 INFO:teuthology.orchestra.run.smithi177.stdout:442 trunc from 0x41eea1a to 0x3931cd7 2024-05-08T03:19:29.250 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:29.249+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:29.260 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.269 INFO:teuthology.orchestra.run.smithi177.stdout:444 write 0x4b2256f thru 0x4b3030e (0xdda0 bytes) 2024-05-08T03:19:29.273 INFO:teuthology.orchestra.run.smithi177.stdout:445 read 0x3ec7647 thru 0x3ec9d0a (0x26c4 bytes) 2024-05-08T03:19:29.273 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:29.273 INFO:teuthology.orchestra.run.smithi177.stdout:448 punch from 0x1b8603d to 0x1b8cb04, (0x6ac7 bytes) 2024-05-08T03:19:29.286 INFO:teuthology.orchestra.run.smithi177.stdout:449 writesame 0xb1acbdc thru 0xb1b6ee3 data_size 0x4a(0xa308 bytes) 2024-05-08T03:19:29.286 INFO:teuthology.orchestra.run.smithi177.stdout:451 clone 4 order 21 su 262144 sc 6 2024-05-08T03:19:29.327 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.327 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.336 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:29.380 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:29.378+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:29.441 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.441 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:29.491 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:29.489+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:29.552 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:29.551+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:29.557 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:29.557 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:29.557 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:29.557 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-05-08T03:19:29.594 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.594 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:29.653 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:29.652+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:29.709 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.709 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:29.763 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:29.762+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:29.830 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone3 from 0xb1b6ee4 (overlap 0x8f6152) to 0x395a20 2024-05-08T03:19:29.845 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:29.845 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:29.855 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:29.853+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:29.858 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:29.901 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:19:29.899+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-05-08T03:19:29.956 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:29.955+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:29.983 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-05-08T03:19:30.016 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:30.016 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:30.025 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:30.034 INFO:teuthology.orchestra.run.smithi177.stdout:452 read 0x6b00499 thru 0x6b06713 (0x627b bytes) 2024-05-08T03:19:30.034 INFO:teuthology.orchestra.run.smithi177.stdout:453 clone 5 order 23 su 131072 sc 13 2024-05-08T03:19:30.058 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:30.056+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:30.138 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:30.139 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:30.147 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:30.191 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:30.190+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:30.254 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:30.254 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:30.260 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:30.258+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:30.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:30.361 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:30.360+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:30.408 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:30.409 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:30.418 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:30.502 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:30.500+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:30.562 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:30.562 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:30.563 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:30.561+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:30.571 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:30.664 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:30.662+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:30.697 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:30.716 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:30.717 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:30.726 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:30.768 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:30.766+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:30.821 INFO:teuthology.orchestra.run.smithi177.stderr:marked out osd.4. 2024-05-08T03:19:30.823 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:19:30.827 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:30.827 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:30.847 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:30.848 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:30.865 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:30.863+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:30.866 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:30.866 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:30.884 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:30.884 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:30.885 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:30.885 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:30.885 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:30.886 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:30.886 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:30.895 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:30.902 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:30.902 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:30.920 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:30.939 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:30.939 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:30.957 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:30.957 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:30.965 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:30.964+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:435 3.4 e23/19 3:22315cfb:::rbd_data.1182b3dbc99f.0000000000000019:head v 23'2, mlcod=0'0)", 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.109764+0000", 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9027198160000003, 2024-05-08T03:19:31.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00062666200000000005, 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109764+0000", 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109764+0000", 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109765+0000", 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.144e-06 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109776+0000", 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0288999999999999e-05 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109792+0000", 2024-05-08T03:19:31.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6608000000000001e-05 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109801+0000", 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9320000000000002e-06 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109873+0000", 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1896000000000004e-05 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.109897+0000", 2024-05-08T03:19:31.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.374e-05 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110346+0000", 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00044950900000000001 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.110391+0000", 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4543999999999999e-05 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.018 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:472 3.0 e23/19 3:02cf0f5e:::rbd_data.1182b3dbc99f.000000000000000c:head v 23'2, mlcod=0'0)", 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.156828+0000", 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.8556557529999997, 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001432689, 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156828+0000", 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156828+0000", 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156829+0000", 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2449999999999999e-06 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156843+0000", 2024-05-08T03:19:31.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3397999999999999e-05 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156863+0000", 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0111000000000001e-05 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156874+0000", 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0689e-05 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156937+0000", 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3864999999999999e-05 2024-05-08T03:19:31.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.156962+0000", 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.499e-05 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.158216+0000", 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0012537959999999999 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.158261+0000", 2024-05-08T03:19:31.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4595000000000002e-05 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:544 3.0 e24/19 3:04479396:::rbd_id.image_client.0-clone2:head v 24'3, mlcod=23'1)", 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.1826289750000001, 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00072031500000000002, 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829855+0000", 2024-05-08T03:19:31.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.68e-07 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829860+0000", 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829872+0000", 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2109999999999999e-05 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829880+0000", 2024-05-08T03:19:31.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9840000000000001e-06 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829939+0000", 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9005999999999997e-05 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.829963+0000", 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4530999999999999e-05 2024-05-08T03:19:31.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.830533+0000", 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00056923499999999997 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:26.830575+0000", 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2734000000000001e-05 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:1356 3.4 e25/19 3:217f9d77:::rbd_data.1182c698972d.00000000000000f1:head v 25'3, mlcod=23'1)", 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.142979+0000", 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.8695045640000001, 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00083450199999999999, 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.142979+0000", 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.142979+0000", 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.142980+0000", 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9999999999999999e-07 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143000+0000", 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0349000000000001e-05 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143011+0000", 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1043000000000001e-05 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143017+0000", 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4969999999999998e-06 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143072+0000", 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4926000000000002e-05 2024-05-08T03:19:31.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143100+0000", 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7997999999999999e-05 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143772+0000", 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00067130599999999999 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.143814+0000", 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2182999999999999e-05 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.031 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:1653 3.9 e25/19 3:9586b532:::rbd_data.1182c698972d.00000000000002f2:head v 25'5, mlcod=22'3)", 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.235709+0000", 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.7767746620000002, 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00084089799999999997, 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235709+0000", 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235709+0000", 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235709+0000", 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4299999999999999e-07 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235749+0000", 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9677e-05 2024-05-08T03:19:31.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235758+0000", 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3829999999999992e-06 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235766+0000", 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7090000000000008e-06 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235816+0000", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9978999999999997e-05 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.235834+0000", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7445e-05 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.236517+0000", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068373499999999999 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.236550+0000", 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2727e-05 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:1700 3.9 e25/19 3:9586b532:::rbd_data.1182c698972d.00000000000002f2:head v 25'6, mlcod=22'4)", 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.283640+0000", 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.728843425, 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00082682700000000003, 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.036 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283640+0000", 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283640+0000", 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283641+0000", 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.95e-07 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283649+0000", 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0430000000000002e-06 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283662+0000", 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3760999999999999e-05 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283668+0000", 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2079999999999997e-06 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283724+0000", 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5716e-05 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.283739+0000", 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4943000000000001e-05 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.284413+0000", 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000674261 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.284467+0000", 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000002e-05 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:1905 3.4 e25/19 3:217f9d77:::rbd_data.1182c698972d.00000000000000f1:head v 25'4, mlcod=23'2)", 2024-05-08T03:19:31.040 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.420892+0000", 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.591592071, 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00066711500000000003, 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420892+0000", 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420892+0000", 2024-05-08T03:19:31.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420893+0000", 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2139999999999999e-06 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420902+0000", 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0129999999999999e-06 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420922+0000", 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9867000000000001e-05 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420932+0000", 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6579999999999997e-06 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.420998+0000", 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6438000000000003e-05 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.421022+0000", 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4114e-05 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.421508+0000", 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00048553799999999999 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.421559+0000", 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1273000000000002e-05 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:2068 3.4 e25/19 3:224e3224:::rbd_data.1182c698972d.00000000000002d5:head v 25'5, mlcod=25'3)", 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.489265+0000", 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.5232192109999998, 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000828746, 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489265+0000", 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489265+0000", 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489266+0000", 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1990000000000001e-06 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489298+0000", 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.235e-05 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489314+0000", 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6137000000000002e-05 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489323+0000", 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0960000000000007e-06 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489378+0000", 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4755e-05 2024-05-08T03:19:31.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.489401+0000", 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3272999999999999e-05 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.490060+0000", 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00065817299999999996 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.490093+0000", 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3763000000000002e-05 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:2100 3.0 e25/19 3:02def5e9:::rbd_data.1182c698972d.00000000000000c9:head v 25'4, mlcod=23'2)", 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.535236+0000", 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.4772480219999999, 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00079425100000000005, 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535236+0000", 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535236+0000", 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535237+0000", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.083e-06 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535264+0000", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.72e-05 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535278+0000", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3648999999999999e-05 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535286+0000", 2024-05-08T03:19:31.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7379999999999994e-06 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535343+0000", 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7225000000000002e-05 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535366+0000", 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2861999999999999e-05 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.535997+0000", 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00063157100000000004 2024-05-08T03:19:31.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.536030+0000", 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2923000000000003e-05 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:2109 3.4 e25/19 3:224e3224:::rbd_data.1182c698972d.00000000000002d5:head v 25'6, mlcod=25'4)", 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.556781+0000", 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.455702708, 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00086202499999999999, 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.053 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556781+0000", 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556781+0000", 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556783+0000", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.482e-06 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556794+0000", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0966e-05 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556813+0000", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9775999999999998e-05 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556822+0000", 2024-05-08T03:19:31.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7630000000000006e-06 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556888+0000", 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6179000000000005e-05 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.556913+0000", 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4890000000000001e-05 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557603+0000", 2024-05-08T03:19:31.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069018599999999997 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557643+0000", 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9783000000000002e-05 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:2103 3.0 e25/19 3:02def5e9:::rbd_data.1182c698972d.00000000000000c9:head v 25'5, mlcod=24'3)", 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.557210+0000", 2024-05-08T03:19:31.057 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 2.4552739180000001, 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00075931200000000005, 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557210+0000", 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557210+0000", 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557210+0000", 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5400000000000002e-07 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557228+0000", 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7604999999999998e-05 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557240+0000", 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2447e-05 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557249+0000", 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7350000000000002e-06 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557302+0000", 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2305999999999997e-05 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557320+0000", 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8270999999999998e-05 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557930+0000", 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00061036599999999999 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.557969+0000", 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9128000000000002e-05 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.062 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:31.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:31.066 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:31.064+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:31.166 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:31.165+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:31.169 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:31.169 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:31.169 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:31.169 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:31.178 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:31.268 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:31.266+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:31.324 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:31.324 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:31.324 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:31.324 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:31.324 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:31.332 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:31.377 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:31.376+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:31.439 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1478 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2461696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158288+0000", 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279222356, 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013760748, 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1478 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158288+0000", 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158288+0000", 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158288+0000", 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158289+0000", 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9199999999999999e-07 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158292+0000", 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9840000000000001e-06 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158294+0000", 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.159e-06 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.171971+0000", 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013676864 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172009+0000", 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8407e-05 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172048+0000", 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9233000000000002e-05 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1479 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158297+0000", 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2792133419999998, 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013936399, 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1479 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158297+0000", 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158297+0000", 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158297+0000", 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158298+0000", 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0699999999999996e-07 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158301+0000", 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9490000000000001e-06 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158306+0000", 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4149999999999998e-06 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172077+0000", 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013771324 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172158+0000", 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309000000000006e-05 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172233+0000", 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5285999999999997e-05 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1480 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2478080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158309+0000", 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279200823, 2024-05-08T03:19:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014032960000000001, 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1480 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158309+0000", 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158309+0000", 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158309+0000", 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3099999999999999e-07 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158313+0000", 2024-05-08T03:19:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.636e-06 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158316+0000", 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1949999999999999e-06 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158318+0000", 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2689999999999998e-06 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172260+0000", 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013942059 2024-05-08T03:19:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172301+0000", 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0559000000000002e-05 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172342+0000", 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1010999999999997e-05 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1481 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2486272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158322+0000", 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279187485, 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014196171, 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1481 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158322+0000", 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158322+0000", 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158323+0000", 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9299999999999999e-07 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158324+0000", 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4500000000000001e-06 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158327+0000", 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0199999999999999e-06 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158330+0000", 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2359999999999999e-06 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172372+0000", 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014042521000000001 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172446+0000", 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3584000000000006e-05 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172519+0000", 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2966999999999994e-05 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1482 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2494464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158332+0000", 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2791774359999999, 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01429156, 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1482 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158332+0000", 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158332+0000", 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158333+0000", 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158334+0000", 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.02e-07 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158337+0000", 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7570000000000002e-06 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158340+0000", 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1919999999999999e-06 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172545+0000", 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014205715000000001 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172582+0000", 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7014000000000002e-05 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172624+0000", 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.176e-05 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1483 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2502656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158342+0000", 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279167637, 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014456439999999999, 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1483 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158342+0000", 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158342+0000", 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158342+0000", 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158343+0000", 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9699999999999994e-07 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158346+0000", 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0230000000000001e-06 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158349+0000", 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.261e-06 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172654+0000", 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014305061000000001 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172727+0000", 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3133000000000006e-05 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172799+0000", 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1742000000000005e-05 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.464 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1484 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2510848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158351+0000", 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2791584149999999, 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014554403, 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1484 2024-05-08T03:19:31.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158351+0000", 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158351+0000", 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158352+0000", 2024-05-08T03:19:31.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9200000000000002e-07 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158353+0000", 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.85e-07 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158356+0000", 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0489999999999999e-06 2024-05-08T03:19:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158358+0000", 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2419999999999999e-06 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172827+0000", 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014469107 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172866+0000", 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8560999999999999e-05 2024-05-08T03:19:31.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172906+0000", 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0266999999999999e-05 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1485 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2519040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158361+0000", 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2791492780000002, 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01471889, 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.472 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1485 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158361+0000", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158361+0000", 2024-05-08T03:19:31.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158361+0000", 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.04e-07 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158363+0000", 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5519999999999999e-06 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158367+0000", 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158369+0000", 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.172936+0000", 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014567138 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173009+0000", 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2662000000000003e-05 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173079+0000", 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0889000000000005e-05 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1486 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2527232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158372+0000", 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2791381529999999, 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014821376000000001, 2024-05-08T03:19:31.476 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1486 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158372+0000", 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158372+0000", 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158372+0000", 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158373+0000", 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.017e-06 2024-05-08T03:19:31.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158376+0000", 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.931e-06 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158382+0000", 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.66e-06 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173113+0000", 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014731031 2024-05-08T03:19:31.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173154+0000", 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0951999999999997e-05 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173193+0000", 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9567000000000003e-05 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1487 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2535424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.480 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158386+0000", 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279123888, 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014983434, 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1487 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158386+0000", 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158386+0000", 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158386+0000", 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158387+0000", 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.08e-06 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158390+0000", 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0819999999999999e-06 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158393+0000", 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173223+0000", 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014830855 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173297+0000", 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3909e-05 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173369+0000", 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2078999999999999e-05 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.484 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1488 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158395+0000", 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279114463, 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015082341000000001, 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1488 2024-05-08T03:19:31.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158395+0000", 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158395+0000", 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158396+0000", 2024-05-08T03:19:31.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158397+0000", 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5000000000000001e-07 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158400+0000", 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.027e-06 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158405+0000", 2024-05-08T03:19:31.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1150000000000002e-06 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173398+0000", 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014992788 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173436+0000", 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8034000000000002e-05 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173478+0000", 2024-05-08T03:19:31.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2218999999999997e-05 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1489 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2551808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158408+0000", 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2791021979999999, 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015161170999999999, 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.489 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1489 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158408+0000", 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158408+0000", 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158408+0000", 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1400000000000001e-07 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158413+0000", 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158416+0000", 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5700000000000001e-06 2024-05-08T03:19:31.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158419+0000", 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3329999999999999e-06 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173494+0000", 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015075531 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173530+0000", 2024-05-08T03:19:31.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5349000000000003e-05 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173569+0000", 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9237999999999999e-05 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1490 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2560000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158422+0000", 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2790879039999998, 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015276672999999999, 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.493 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1490 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158422+0000", 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158422+0000", 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158422+0000", 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6800000000000001e-07 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158423+0000", 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6099999999999999e-07 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158426+0000", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9100000000000001e-06 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158429+0000", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2859999999999998e-06 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173602+0000", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015173367 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173650+0000", 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7772000000000003e-05 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173699+0000", 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9008999999999998e-05 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1491 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2568192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158438+0000", 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.279071847, 2024-05-08T03:19:31.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015354292, 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1491 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158438+0000", 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158438+0000", 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158438+0000", 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3799999999999999e-07 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158443+0000", 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9520000000000002e-06 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158447+0000", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6780000000000002e-06 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158449+0000", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4420000000000001e-06 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173714+0000", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015264318000000001 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173752+0000", 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8546e-05 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173792+0000", 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0117999999999998e-05 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1492 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2576384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158453+0000", 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.27905705, 2024-05-08T03:19:31.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015455032, 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1492 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158453+0000", 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158453+0000", 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158453+0000", 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1399999999999998e-07 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158454+0000", 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.017e-06 2024-05-08T03:19:31.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158458+0000", 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9330000000000003e-06 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158461+0000", 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4210000000000002e-06 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173819+0000", 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01535806 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173860+0000", 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1461999999999997e-05 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173908+0000", 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7824999999999997e-05 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.505 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1493 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2584576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158464+0000", 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2790462050000002, 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015641693000000002, 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1493 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158464+0000", 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158464+0000", 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158464+0000", 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1e-07 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158465+0000", 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8299999999999995e-07 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158468+0000", 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9739999999999998e-06 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158470+0000", 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1500000000000002e-06 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.173945+0000", 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015475022 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174021+0000", 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5605999999999994e-05 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174105+0000", 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4747999999999996e-05 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.509 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1494 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2592768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158473+0000", 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2790366980000001, 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015738783999999999, 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.510 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1494 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158473+0000", 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158473+0000", 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158473+0000", 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158474+0000", 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1800000000000004e-07 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158477+0000", 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.976e-06 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158480+0000", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2369999999999999e-06 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174133+0000", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015653369 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174171+0000", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7762999999999997e-05 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174212+0000", 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1309e-05 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1495 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2600960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2790276490000001, 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015832629000000001, 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.514 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1495 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.17e-07 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158483+0000", 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9599999999999998e-07 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158486+0000", 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9450000000000002e-06 2024-05-08T03:19:31.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158488+0000", 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174237+0000", 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015748306 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174275+0000", 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8578999999999998e-05 2024-05-08T03:19:31.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174315+0000", 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9495e-05 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1496 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2609152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2790187400000002, 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015926348999999999, 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.518 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1496 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158492+0000", 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3600000000000002e-07 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158495+0000", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8339999999999999e-06 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158497+0000", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2230000000000001e-06 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174338+0000", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015840481 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174376+0000", 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8642e-05 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174417+0000", 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1025000000000001e-05 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1497 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:31.522 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.2790068510000001, 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016000936, 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1497 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3999999999999998e-07 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158504+0000", 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.071e-06 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158508+0000", 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3440000000000001e-06 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158513+0000", 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7899999999999996e-06 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174433+0000", 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015919098999999999 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174468+0000", 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5447999999999997e-05 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174504+0000", 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5944e-05 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:31.526 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:31.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:31.528 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:31.526+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:31.629 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:31.628+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:31.644 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:31.644 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:31.644 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:31.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:31.730 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:31.729+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:31.759 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:31.760 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:31.760 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:31.760 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:31.760 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:31.769 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:31.824 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:19:31.830 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:31.829+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0187338669999999, 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.889 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:31.894 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0183916890000004, 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.895 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.899 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.018352857, 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:19:31.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:19:31.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:31.905 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0183068239999997, 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:31.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:19:31.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:19:31.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:19:31.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:19:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0172896319999998, 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.911 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.915 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.916 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0172533809999997, 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:19:31.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:19:31.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:19:31.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:19:31.921 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0172260819999996, 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.922 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.927 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0171963890000004, 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:31.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:19:31.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:19:31.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:19:31.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:19:31.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0171708739999996, 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.933 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:31.938 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0070567080000004, 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:31.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:19:31.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:19:31.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:19:31.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:19:31.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7547178629999998, 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:19:31.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7546995320000001, 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.948 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.952 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7546750709999999, 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7546487280000003, 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.957 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:31.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:19:31.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:19:31.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7546313370000002, 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.961 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7546140559999999, 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.965 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:31.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:19:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:19:31.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7545866189999999, 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.969 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:19:31.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7545686599999999, 2024-05-08T03:19:31.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:31.977 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.7545515739999997, 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:31.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:19:31.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:19:31.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:31.981 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.75453459, 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:31.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:31.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:31.986 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:31.986 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:31.987 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:31.988 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:31.985+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:32.088 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:32.087+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:32.132 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:32.132 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:32.132 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:32.132 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:32.140 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:32.189 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:32.187+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:32.246 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:32.246 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:32.246 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:32.246 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:32.246 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:32.255 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:32.290 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:32.288+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:32.391 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:32.389+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:22.784831+0000", 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.6144661280000001, 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035186200000000001, 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 10 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784831+0000", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784831+0000", 2024-05-08T03:19:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784832+0000", 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.094e-06 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784836+0000", 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6830000000000001e-06 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784843+0000", 2024-05-08T03:19:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0219999999999999e-06 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784850+0000", 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9439999999999999e-06 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784904+0000", 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4669000000000001e-05 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:19:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.784910+0000", 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7139999999999998e-06 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.786371+0000", 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001461157 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.786459+0000", 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7656000000000002e-05 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.786615+0000", 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00015603000000000001 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.787718+0000", 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0011030079999999999 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.788257+0000", 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053960300000000004 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.788304+0000", 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6532000000000001e-05 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:22.788349+0000", 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5507999999999999e-05 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:695 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.807810+0000", 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.591487367, 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033177509999999999, 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 695 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807810+0000", 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807810+0000", 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807811+0000", 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.798e-06 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807820+0000", 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807837+0000", 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6716000000000001e-05 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807855+0000", 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8321e-05 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807928+0000", 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3481999999999999e-05 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.807937+0000", 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3869999999999995e-06 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.809711+0000", 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001774503 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.809809+0000", 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7604999999999998e-05 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.809900+0000", 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0777000000000002e-05 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.810643+0000", 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00074296900000000001 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.811017+0000", 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037460299999999999 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.811079+0000", 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1424e-05 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.811127+0000", 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8686000000000001e-05 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:742 3.5 3:a7daa15d:::rbd_id.image_client.0-clone3:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.839731+0000", 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5595660440000003, 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021684590000000002, 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 742 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839731+0000", 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839731+0000", 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839732+0000", 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5000000000000001e-07 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839737+0000", 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1510000000000003e-06 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839747+0000", 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9040000000000004e-06 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839752+0000", 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4859999999999997e-06 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839801+0000", 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9052999999999999e-05 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.839894+0000", 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3029000000000007e-05 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.840025+0000", 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013101200000000001 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.840956+0000", 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00093018599999999995 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.841807+0000", 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00085113799999999998 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.841858+0000", 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1109999999999997e-05 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.841899+0000", 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1539999999999999e-05 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:809 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867229+0000", 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5320677209999998, 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051126690000000002, 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 809 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867229+0000", 2024-05-08T03:19:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867229+0000", 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867231+0000", 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6500000000000001e-06 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867241+0000", 2024-05-08T03:19:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0105000000000001e-05 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867260+0000", 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8998999999999999e-05 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867270+0000", 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6879999999999994e-06 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867347+0000", 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7312999999999997e-05 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867451+0000", 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000104497 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867818+0000", 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00036685400000000001 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871227+0000", 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0034083429999999999 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872232+0000", 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010048920000000001 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872292+0000", 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0479000000000002e-05 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872342+0000", 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9849000000000003e-05 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:811 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867440+0000", 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5318572420000001, 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0054472629999999999, 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 811 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867440+0000", 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867440+0000", 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867441+0000", 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1909999999999999e-06 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867448+0000", 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6109999999999996e-06 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867470+0000", 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1846000000000001e-05 2024-05-08T03:19:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867479+0000", 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9740000000000008e-06 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868259+0000", 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00077978200000000002 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868339+0000", 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0024999999999994e-05 2024-05-08T03:19:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868693+0000", 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035363199999999999 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871233+0000", 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002540596 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872789+0000", 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015553520000000001 2024-05-08T03:19:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872846+0000", 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7469999999999997e-05 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872887+0000", 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0784000000000001e-05 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:814 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867864+0000", 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5314328220000002, 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.005220588, 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 814 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867864+0000", 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867864+0000", 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867865+0000", 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6099999999999999e-07 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867871+0000", 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3219999999999998e-06 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867883+0000", 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.182e-05 2024-05-08T03:19:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867891+0000", 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4200000000000001e-06 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869127+0000", 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00123649 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869176+0000", 2024-05-08T03:19:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8715999999999999e-05 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869590+0000", 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00041444499999999999 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872104+0000", 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025133880000000001 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872971+0000", 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00086717800000000002 2024-05-08T03:19:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873039+0000", 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8115999999999996e-05 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873085+0000", 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5732000000000002e-05 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:817 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868028+0000", 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5312684179999998, 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0052427710000000002, 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 817 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868028+0000", 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868028+0000", 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868029+0000", 2024-05-08T03:19:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2799999999999995e-07 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868035+0000", 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8799999999999996e-06 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868046+0000", 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1069e-05 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.123e-06 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869995+0000", 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0019407649999999999 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870090+0000", 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5218999999999998e-05 2024-05-08T03:19:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870381+0000", 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000291033 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872479+0000", 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002098005 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873159+0000", 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00067927600000000003 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873222+0000", 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3799000000000004e-05 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873271+0000", 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8773999999999998e-05 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:819 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868382+0000", 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5309147970000003, 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050607220000000001, 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 819 2024-05-08T03:19:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868382+0000", 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868382+0000", 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868383+0000", 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7300000000000004e-07 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868390+0000", 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6059999999999999e-06 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868407+0000", 2024-05-08T03:19:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7501000000000001e-05 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868414+0000", 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2729999999999997e-06 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870665+0000", 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002251018 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870715+0000", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9783000000000001e-05 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870953+0000", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023733599999999999 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872503+0000", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001550842 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873336+0000", 2024-05-08T03:19:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00083225200000000001 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873392+0000", 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6706000000000002e-05 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873443+0000", 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0432000000000001e-05 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:822 3.1d 3:ba31a001:::rbd_object_map.11821677beed.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868928+0000", 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.530369114, 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0046729629999999996, 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 822 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868928+0000", 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868928+0000", 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868929+0000", 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3799999999999996e-07 2024-05-08T03:19:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868933+0000", 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3560000000000003e-06 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868946+0000", 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2942000000000001e-05 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868951+0000", 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6180000000000003e-06 2024-05-08T03:19:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871252+0000", 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0023010359999999998 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871298+0000", 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5605000000000001e-05 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871497+0000", 2024-05-08T03:19:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000199347 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872509+0000", 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001011149 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873501+0000", 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00099259400000000007 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873554+0000", 2024-05-08T03:19:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2442999999999999e-05 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873601+0000", 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7135000000000002e-05 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:832 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:head [zero 4194304~2097152] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.876593+0000", 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5227039109999998, 2024-05-08T03:19:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001679346, 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 832 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876593+0000", 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876593+0000", 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876593+0000", 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876596+0000", 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.198e-06 2024-05-08T03:19:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876600+0000", 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9859999999999996e-06 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876605+0000", 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6659999999999996e-06 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877368+0000", 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00076278999999999995 2024-05-08T03:19:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877431+0000", 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3085999999999999e-05 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878272+0000", 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00084120400000000002 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:836 3.1c 3:392975bd:::rbd_data.11821677beed.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879042+0000", 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5202545819999997, 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021920920000000001, 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 836 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879042+0000", 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879042+0000", 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879043+0000", 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6599999999999995e-07 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879116+0000", 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3277999999999999e-05 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879128+0000", 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1311000000000001e-05 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879133+0000", 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.57e-06 2024-05-08T03:19:32.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879193+0000", 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999999e-05 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879349+0000", 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00015587399999999999 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879584+0000", 2024-05-08T03:19:32.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023508599999999999 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880908+0000", 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013233100000000001 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881176+0000", 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026817699999999998 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881209+0000", 2024-05-08T03:19:32.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3206e-05 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881234+0000", 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5403999999999999e-05 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:837 3.1c 3:392975bd:::rbd_data.11821677beed.0000000000000009:head [zero 3353440~1889440] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879149+0000", 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.5201483429999998, 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025564429999999998, 2024-05-08T03:19:32.465 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 837 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879149+0000", 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879149+0000", 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879149+0000", 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879151+0000", 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9700000000000002e-06 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879156+0000", 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7770000000000002e-06 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879159+0000", 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.179e-06 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880474+0000", 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001315129 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880573+0000", 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9266999999999997e-05 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881705+0000", 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0011317230000000001 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.469 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1654 3.1d 3:b810ec49:::rbd_data.1182c698972d.00000000000002f3:head [set-alloc-hint object_size 262144 write_size 262144,write 222144~23728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.235571+0000", 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.1637263549999997, 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016921180000000001, 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1654 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235571+0000", 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235571+0000", 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235572+0000", 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5340000000000001e-06 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235601+0000", 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9082999999999999e-05 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235620+0000", 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9256e-05 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235631+0000", 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0370999999999999e-05 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235681+0000", 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0158999999999999e-05 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235764+0000", 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2820999999999999e-05 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.235847+0000", 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3386000000000004e-05 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.237068+0000", 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012204220000000001 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.237194+0000", 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012629899999999999 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.237229+0000", 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5567e-05 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.237263+0000", 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3219999999999997e-05 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1698 3.1d 3:bffef8b2:::rbd_data.1182c698972d.000000000000022a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.283107+0000", 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.1161899670000004, 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050557220000000003, 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.475 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1698 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283107+0000", 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283107+0000", 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283108+0000", 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.15e-06 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283112+0000", 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283123+0000", 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1394e-05 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283130+0000", 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9689999999999997e-06 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283190+0000", 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9859999999999999e-05 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283267+0000", 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7058000000000007e-05 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283367+0000", 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9969000000000005e-05 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.284259+0000", 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000892145 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.288064+0000", 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003804408 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.288119+0000", 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.558e-05 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.288163+0000", 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3373e-05 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1701 3.1d 3:b810ec49:::rbd_data.1182c698972d.00000000000002f3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.283230+0000", 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.1160672109999998, 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050184699999999997, 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1701 2024-05-08T03:19:32.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283230+0000", 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283230+0000", 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283231+0000", 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283235+0000", 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7049999999999998e-06 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283246+0000", 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0844e-05 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283253+0000", 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1520000000000003e-06 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283673+0000", 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00041968099999999998 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283745+0000", 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2528999999999996e-05 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283849+0000", 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010367800000000001 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.284641+0000", 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00079211699999999995 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.288193+0000", 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035518920000000001 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.288223+0000", 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9374e-05 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.288248+0000", 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5635000000000001e-05 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1909 3.16 3:68f2f734:::rbd_data.1182c698972d.00000000000002bd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.486 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.421842+0000", 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.977454549, 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035705049999999999, 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.487 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.489 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.489 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1909 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421842+0000", 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.490 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421842+0000", 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421843+0000", 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3799999999999997e-07 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421847+0000", 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421855+0000", 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0860000000000002e-06 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421859+0000", 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7949999999999998e-06 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421905+0000", 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630999999999997e-05 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.421956+0000", 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1168000000000002e-05 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.422045+0000", 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8875999999999994e-05 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.422565+0000", 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000519452 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.425313+0000", 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0027485460000000001 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.425364+0000", 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0837999999999997e-05 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.425413+0000", 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8918999999999997e-05 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2042 3.16 3:6914e236:::rbd_data.1182c698972d.0000000000000023:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 241664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.450778+0000", 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.9485189209999998, 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025089940000000001, 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2042 2024-05-08T03:19:32.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450778+0000", 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450778+0000", 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450780+0000", 2024-05-08T03:19:32.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.517e-06 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450786+0000", 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.968e-06 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450801+0000", 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4953000000000001e-05 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450810+0000", 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8389999999999996e-06 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450867+0000", 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6521999999999999e-05 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.450970+0000", 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010354299999999999 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.451086+0000", 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000115532 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.451552+0000", 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046651899999999998 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.453210+0000", 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016573619999999999 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.453245+0000", 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4873000000000003e-05 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.453287+0000", 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2366000000000001e-05 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2086 3.5 3:a32190d3:::rbd_data.1182c698972d.0000000000000040:head [set-alloc-hint object_size 262144 write_size 262144,write 0~1013] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.500 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.517530+0000", 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.8817674000000002, 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001731135, 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2086 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517530+0000", 2024-05-08T03:19:32.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517530+0000", 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517531+0000", 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4580000000000001e-06 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517540+0000", 2024-05-08T03:19:32.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1160000000000005e-06 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517554+0000", 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3651e-05 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517563+0000", 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0410000000000003e-06 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517639+0000", 2024-05-08T03:19:32.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6536e-05 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517728+0000", 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8498999999999994e-05 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517875+0000", 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00014694299999999999 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.518894+0000", 2024-05-08T03:19:32.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001019633 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519164+0000", 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026928199999999997 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519215+0000", 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1779e-05 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519261+0000", 2024-05-08T03:19:32.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5197000000000001e-05 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2138 3.1d 3:bcaa2107:::rbd_data.1182c698972d.0000000000000095:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.587631+0000", 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.8116655389999998, 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0059967739999999999, 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.506 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2138 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587631+0000", 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587631+0000", 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587633+0000", 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3850000000000001e-06 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587639+0000", 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4829999999999996e-06 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587656+0000", 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6994e-05 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587664+0000", 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6260000000000003e-06 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587720+0000", 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6450999999999999e-05 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.587812+0000", 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1372999999999994e-05 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.588783+0000", 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00097104999999999999 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593554+0000", 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0047716629999999998 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593565+0000", 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1037e-05 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593598+0000", 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2258000000000002e-05 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593628+0000", 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0454e-05 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop_reply(client.4482.0:2138 3.1d e25/19 ondisk, result = 0)", 2024-05-08T03:19:32.511 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.591440+0000", 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.8078564579999998, 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021185790000000002, 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.591440+0000", 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.591440+0000", 2024-05-08T03:19:32.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.591441+0000", 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1399999999999997e-07 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.591444+0000", 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2109999999999998e-06 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.591453+0000", 2024-05-08T03:19:32.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6309999999999992e-06 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.591458+0000", 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6e-06 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593546+0000", 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020882280000000001 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593551+0000", 2024-05-08T03:19:32.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.271e-06 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.593559+0000", 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0239999999999997e-06 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.515 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:32.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:32.517 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:32.515+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:32.617 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:32.616+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:32.665 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:32.665 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:32.665 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:32.665 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:32.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:32.717 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:32.716+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:32.777 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:32.778 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:32.778 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:32.778 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:32.778 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:32.787 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:32.818 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:32.817+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:32.895 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:498 3.10 3:0ec1abea:::rbd_object_map.1182b3dbc99f.0000000000000004:head [writefull 0~37] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:26.800203+0000", 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 6.0919503280000002, 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026895220000000002, 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 498 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800203+0000", 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800203+0000", 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800205+0000", 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2109999999999999e-06 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800213+0000", 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7369999999999995e-06 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800227+0000", 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3441999999999999e-05 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800234+0000", 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1840000000000002e-06 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800289+0000", 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4765999999999997e-05 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.800293+0000", 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2289999999999997e-06 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.801804+0000", 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015108840000000001 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.801896+0000", 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1935000000000007e-05 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.801981+0000", 2024-05-08T03:19:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5314000000000003e-05 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.802701+0000", 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00071967699999999995 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.802831+0000", 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012990500000000001 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.802867+0000", 2024-05-08T03:19:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6343000000000001e-05 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.802893+0000", 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5894999999999999e-05 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:562 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:26.844465+0000", 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 6.0476884249999996, 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001082083, 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 562 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844465+0000", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844465+0000", 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844466+0000", 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9999999999999996e-07 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844471+0000", 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2889999999999997e-06 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844482+0000", 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0507e-05 2024-05-08T03:19:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844490+0000", 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9130000000000003e-06 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844536+0000", 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6007999999999997e-05 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844588+0000", 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2117000000000003e-05 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.844720+0000", 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000131939 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.845177+0000", 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045748400000000003 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.845486+0000", 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00030873700000000001 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.845520+0000", 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4241e-05 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.845547+0000", 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6948e-05 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:578 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:26.852974+0000", 2024-05-08T03:19:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 6.0391795960000003, 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00094886600000000003, 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 578 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.852974+0000", 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.852974+0000", 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.852975+0000", 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3799999999999996e-07 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.852979+0000", 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0740000000000003e-06 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.852990+0000", 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0739e-05 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.852995+0000", 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0599999999999998e-06 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853037+0000", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2271000000000002e-05 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853086+0000", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9245e-05 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853191+0000", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000104348 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853644+0000", 2024-05-08T03:19:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045348699999999998 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853865+0000", 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00022143099999999999 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853898+0000", 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2722000000000003e-05 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:26.853923+0000", 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4751000000000001e-05 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:737 3.10 3:0ec1abea:::rbd_object_map.1182b3dbc99f.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.833266+0000", 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.0588875729999998, 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021566559999999999, 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 737 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833266+0000", 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833266+0000", 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833268+0000", 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6819999999999999e-06 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833277+0000", 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3989999999999993e-06 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833294+0000", 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6807e-05 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833305+0000", 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1048e-05 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833373+0000", 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8330999999999993e-05 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.833378+0000", 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8879999999999996e-06 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.835172+0000", 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001793848 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.835255+0000", 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3121000000000006e-05 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.835423+0000", 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00016753199999999999 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:760 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.857657+0000", 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.0344968120000004, 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012983339999999999, 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 760 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857657+0000", 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857657+0000", 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857659+0000", 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6750000000000001e-06 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857666+0000", 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3100000000000003e-06 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857683+0000", 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7448e-05 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857691+0000", 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5059999999999998e-06 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857750+0000", 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9052999999999999e-05 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857809+0000", 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8835999999999997e-05 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.857967+0000", 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000158231 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.858566+0000", 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000598798 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.858886+0000", 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032038100000000001 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.858921+0000", 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5401000000000002e-05 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.858955+0000", 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3695000000000002e-05 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:1252 3.1e e25/19 3:79829b05:::rbd_data.1182c698972d.00000000000001e4:head v 25'2, mlcod=0'0)", 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.108442+0000", 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.7837119560000003, 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098169199999999994, 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108442+0000", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108442+0000", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108443+0000", 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1570000000000001e-06 2024-05-08T03:19:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108501+0000", 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7949999999999999e-05 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108513+0000", 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.216e-05 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108519+0000", 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3489999999999997e-06 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108564+0000", 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4957999999999999e-05 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.108580+0000", 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5692000000000002e-05 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.109371+0000", 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00079122899999999996 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.109423+0000", 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2197000000000002e-05 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1270 3.a 3:51bc2795:::rbd_data.1182c698972d.00000000000001bc:head [set-alloc-hint object_size 262144 write_size 262144,write 0~14839] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.115737+0000", 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.776416556, 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040114210000000003, 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1270 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115737+0000", 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115737+0000", 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115741+0000", 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4570000000000001e-06 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115778+0000", 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7438000000000003e-05 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115794+0000", 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5752000000000001e-05 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115802+0000", 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.738e-06 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115866+0000", 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3041999999999997e-05 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.115989+0000", 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012312400000000001 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.116097+0000", 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010878 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119239+0000", 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0031411880000000001 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119686+0000", 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00044713700000000001 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119719+0000", 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3016999999999999e-05 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119749+0000", 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9748e-05 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1271 3.a 3:564e02ff:::rbd_data.1182c698972d.00000000000001bd:head [set-alloc-hint object_size 262144 write_size 262144,write 0~12288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.117182+0000", 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.7749718650000004, 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026752099999999999, 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1271 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117182+0000", 2024-05-08T03:19:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117182+0000", 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117183+0000", 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117200+0000", 2024-05-08T03:19:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7422999999999999e-05 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117210+0000", 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0414999999999999e-05 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117220+0000", 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1679999999999993e-06 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117264+0000", 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4888000000000001e-05 2024-05-08T03:19:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117351+0000", 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6831999999999998e-05 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.117459+0000", 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000107398 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119633+0000", 2024-05-08T03:19:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021739239999999998 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119783+0000", 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000150488 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119833+0000", 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9675000000000001e-05 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.119857+0000", 2024-05-08T03:19:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4223000000000002e-05 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1517 3.a 3:5513f810:::rbd_data.1182c698972d.00000000000001f4:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.194299+0000", 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.6978543269999999, 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012197760000000001, 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1517 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194299+0000", 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194299+0000", 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194301+0000", 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6559999999999999e-06 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194310+0000", 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.636e-06 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194325+0000", 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4965e-05 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194334+0000", 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3959999999999995e-06 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194386+0000", 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2193e-05 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194493+0000", 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000106477 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.194617+0000", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000124349 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.195389+0000", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00077194299999999998 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.195415+0000", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5709000000000001e-05 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.195472+0000", 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7297999999999999e-05 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.195519+0000", 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7154000000000003e-05 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1684 3.a 3:51bc2795:::rbd_data.1182c698972d.00000000000001bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.278003+0000", 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.614150897, 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0049509080000000004, 2024-05-08T03:19:32.945 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1684 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278003+0000", 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278003+0000", 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278004+0000", 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7320000000000001e-06 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278010+0000", 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4530000000000001e-06 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278023+0000", 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3396e-05 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278030+0000", 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6640000000000001e-06 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278093+0000", 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3278e-05 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278184+0000", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1032999999999994e-05 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278315+0000", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013050400000000001 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.279985+0000", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016702030000000001 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282882+0000", 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028966819999999998 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282925+0000", 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2953e-05 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282954+0000", 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9009999999999998e-05 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1685 3.a 3:564e02ff:::rbd_data.1182c698972d.00000000000001bd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.278048+0000", 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.6141061140000001, 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050107179999999999, 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1685 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278048+0000", 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278048+0000", 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278048+0000", 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1699999999999999e-07 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278050+0000", 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278056+0000", 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8590000000000001e-06 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278060+0000", 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5700000000000003e-06 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278667+0000", 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000606593 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278751+0000", 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4138e-05 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278859+0000", 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010826 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282677+0000", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038173669999999999 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282996+0000", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031895699999999998 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283035+0000", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9508000000000002e-05 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283058+0000", 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3110000000000001e-05 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1688 3.a 3:5513f810:::rbd_data.1182c698972d.00000000000001f4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.278067+0000", 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.6140863459999997, 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050852359999999999, 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1688 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278067+0000", 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278067+0000", 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278068+0000", 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.41e-07 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278070+0000", 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.328e-06 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278077+0000", 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7109999999999998e-06 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278082+0000", 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.152e-06 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.279092+0000", 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010098209999999999 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.279175+0000", 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3619e-05 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.279251+0000", 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5877999999999994e-05 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282708+0000", 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003456612 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283093+0000", 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00038488800000000001 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283118+0000", 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5199e-05 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.283153+0000", 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4687000000000002e-05 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:1686 3.1e e25/19 3:79829b05:::rbd_data.1182c698972d.00000000000001e4:head v 25'3, mlcod=23'1)", 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.278536+0000", 2024-05-08T03:19:32.961 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.6136178929999998, 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042138690000000003, 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278536+0000", 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278536+0000", 2024-05-08T03:19:32.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278537+0000", 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6899999999999996e-07 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278544+0000", 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8900000000000001e-06 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278553+0000", 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7690000000000008e-06 2024-05-08T03:19:32.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278607+0000", 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3792999999999998e-05 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278948+0000", 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034104799999999999 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.278968+0000", 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0366000000000001e-05 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282706+0000", 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037378979999999999 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.282750+0000", 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3235999999999997e-05 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.965 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2064 3.10 3:0b68ad67:::rbd_data.1182c698972d.0000000000000379:head [set-alloc-hint object_size 262144 write_size 262144,write 159744~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.483183+0000", 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.408970719, 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001434846, 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2064 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483183+0000", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483183+0000", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483185+0000", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4660000000000002e-06 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483200+0000", 2024-05-08T03:19:32.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4266e-05 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483220+0000", 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9972000000000001e-05 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483230+0000", 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9590000000000007e-06 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483296+0000", 2024-05-08T03:19:32.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6909000000000006e-05 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483457+0000", 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000160431 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.483595+0000", 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000137713 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.484418+0000", 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00082364700000000005 2024-05-08T03:19:32.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.484561+0000", 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00014244999999999999 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.484592+0000", 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0920999999999998e-05 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.484618+0000", 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6112e-05 2024-05-08T03:19:32.970 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:2072 3.1e e25/19 3:7aa9d5a6:::rbd_data.1182c698972d.000000000000014b:head v 25'4, mlcod=25'2)", 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.492666+0000", 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.3994878020000003, 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00082817000000000003, 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492666+0000", 2024-05-08T03:19:32.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492666+0000", 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492667+0000", 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5799999999999999e-07 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492676+0000", 2024-05-08T03:19:32.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7820000000000005e-06 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492685+0000", 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1619999999999997e-06 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492692+0000", 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2269999999999998e-06 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492752+0000", 2024-05-08T03:19:32.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0525000000000001e-05 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.492777+0000", 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4536999999999999e-05 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.493451+0000", 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00067464499999999998 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.493494+0000", 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2633999999999999e-05 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2084 3.a 3:5362fa90:::rbd_data.1182c698972d.000000000000003e:head [set-alloc-hint object_size 262144 write_size 262144,write 237568~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.517154+0000", 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.3749996949999996, 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021582300000000001, 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.975 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2084 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517154+0000", 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517154+0000", 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517156+0000", 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5549999999999999e-06 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517195+0000", 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9745e-05 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517213+0000", 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7254000000000001e-05 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517223+0000", 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0594000000000001e-05 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517291+0000", 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8337999999999995e-05 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517468+0000", 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017677799999999999 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517607+0000", 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000138543 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.518529+0000", 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00092248899999999997 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519259+0000", 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00072958799999999998 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519288+0000", 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9057e-05 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519312+0000", 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4289000000000001e-05 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2085 3.10 3:0d9a1e29:::rbd_data.1182c698972d.000000000000003f:head [set-alloc-hint object_size 262144 write_size 262144,write 237149~24995] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.980 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.517251+0000", 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.374902928, 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018978689999999999, 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2085 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517251+0000", 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517251+0000", 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517251+0000", 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517265+0000", 2024-05-08T03:19:32.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3694e-05 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517273+0000", 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7130000000000005e-06 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517278+0000", 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6330000000000001e-06 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517322+0000", 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3355000000000001e-05 2024-05-08T03:19:32.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517446+0000", 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000124187 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.517551+0000", 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000105608 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.518762+0000", 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012104450000000001 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519044+0000", 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00028214599999999999 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519098+0000", 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4422000000000002e-05 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.519149+0000", 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0223999999999999e-05 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:2105 3.1e e25/19 3:7aa9d5a6:::rbd_data.1182c698972d.000000000000014b:head v 25'6, mlcod=25'4)", 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.556956+0000", 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.3351975400000002, 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00091365800000000003, 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.556956+0000", 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.556956+0000", 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.556957+0000", 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.556965+0000", 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8630000000000008e-06 2024-05-08T03:19:32.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.556979+0000", 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4335000000000001e-05 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.556986+0000", 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7429999999999996e-06 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.557048+0000", 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1731999999999998e-05 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.557069+0000", 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1186e-05 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.557833+0000", 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000763991 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.557870+0000", 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6838999999999997e-05 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.989 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:2115 3.10 3:0b68ad67:::rbd_data.1182c698972d.0000000000000379:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.558423+0000", 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.3337304950000002, 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002360343, 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 2115 2024-05-08T03:19:32.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558423+0000", 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558423+0000", 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558425+0000", 2024-05-08T03:19:32.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3740000000000001e-06 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558431+0000", 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6490000000000002e-06 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558446+0000", 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.471e-05 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558455+0000", 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3379999999999994e-06 2024-05-08T03:19:32.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558523+0000", 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7411999999999998e-05 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558601+0000", 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8688000000000003e-05 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558706+0000", 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000104515 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.559478+0000", 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00077252799999999997 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.560719+0000", 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001240641 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.560753+0000", 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3970999999999997e-05 2024-05-08T03:19:32.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.560783+0000", 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0516999999999999e-05 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:2113 3.1e e25/19 3:7ea1e23b:::rbd_data.1182c698972d.0000000000000357:head v 25'7, mlcod=25'5)", 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.558601+0000", 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.3335522639999997, 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001943757, 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:32.995 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558601+0000", 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558601+0000", 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:32.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558602+0000", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558607+0000", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.169e-06 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558616+0000", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9379999999999998e-06 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558622+0000", 2024-05-08T03:19:32.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0039999999999996e-06 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558667+0000", 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4708e-05 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.558686+0000", 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9351000000000002e-05 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.560506+0000", 2024-05-08T03:19:32.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001820407 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.560545+0000", 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8671999999999998e-05 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:32.999 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:33.000 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:33.001 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:32.999+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:33.002 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:33.031 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone4 from 0xb1b6ee4 (overlap 0xb1b6ee4) to 0x4715f8e 2024-05-08T03:19:33.103 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:33.100+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:33.119 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:33.119 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:33.119 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:33.119 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:33.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:33.153 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:33.154 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:33.172 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:33.172 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:33.192 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:33.193 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:33.203 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:33.202+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:33.204 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-05-08T03:19:33.219 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:33.219 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:33.249 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:33.249 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:33.267 INFO:teuthology.orchestra.run.smithi177.stdout:454 punch from 0x9593476 to 0x9597541, (0x40cb bytes) 2024-05-08T03:19:33.269 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:33.269 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:33.271 INFO:teuthology.orchestra.run.smithi177.stdout:456 read 0x4d7a8ad thru 0x4d84963 (0xa0b7 bytes) 2024-05-08T03:19:33.271 INFO:teuthology.orchestra.run.smithi177.stdout:457 writesame 0x8f98bb7 thru 0x8fa6754 data_size 0x1(0xdb9e bytes) 2024-05-08T03:19:33.288 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:33.288 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:33.288 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:33.288 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:33.288 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:33.289 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:33.289 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:33.293 INFO:teuthology.orchestra.run.smithi177.stdout:458 writesame 0x211de86 thru 0x2123f99 data_size 0x26(0x6114 bytes) 2024-05-08T03:19:33.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:33.306 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:33.304+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:33.316 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:33.316 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:33.345 INFO:teuthology.orchestra.run.smithi177.stdout:459 read 0x342f889 thru 0x343785b (0x7fd3 bytes) 2024-05-08T03:19:33.346 INFO:teuthology.orchestra.run.smithi177.stdout:461 write 0x295f30c thru 0x2965f15 (0x6c0a bytes) 2024-05-08T03:19:33.386 INFO:teuthology.orchestra.run.smithi177.stdout:462 write 0x4d3518e thru 0x4d36306 (0x1179 bytes) 2024-05-08T03:19:33.388 INFO:teuthology.orchestra.run.smithi177.stdout:463 write 0x287d3bf thru 0x2880481 (0x30c3 bytes) 2024-05-08T03:19:33.406 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:33.405+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:33.414 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:33.414 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:33.423 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:33.426 INFO:teuthology.orchestra.run.smithi177.stdout:464 writesame 0x410486d thru 0x41099e8 data_size 0x23(0x517c bytes) 2024-05-08T03:19:33.482 INFO:teuthology.orchestra.run.smithi177.stdout:465 read 0x7e9ace9 thru 0x7ea5b80 (0xae98 bytes) 2024-05-08T03:19:33.482 INFO:teuthology.orchestra.run.smithi177.stdout:466 write 0x374acb9 thru 0x3757879 (0xcbc1 bytes) 2024-05-08T03:19:33.484 INFO:teuthology.orchestra.run.smithi177.stdout:467 read 0x8495041 thru 0x849fd9d (0xad5d bytes) 2024-05-08T03:19:33.484 INFO:teuthology.orchestra.run.smithi177.stdout:468 write 0x7c18c35 thru 0x7c1cabc (0x3e88 bytes) 2024-05-08T03:19:33.495 INFO:teuthology.orchestra.run.smithi177.stdout:469 write 0xda97d77 thru 0xdaa0967 (0x8bf1 bytes) 2024-05-08T03:19:33.502 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:33.507 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:33.505+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:33.512 INFO:teuthology.orchestra.run.smithi177.stdout:471 trunc from 0xdaa0968 to 0x4c2bca6 2024-05-08T03:19:33.559 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:33.559 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:33.574 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:33.608 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:33.607+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:33.624 INFO:teuthology.orchestra.run.smithi177.stdout:474 read 0x2a1e4b8 thru 0x2a1e85a (0x3a3 bytes) 2024-05-08T03:19:33.624 INFO:teuthology.orchestra.run.smithi177.stdout:476 read 0x4c254c2 thru 0x4c292ab (0x3dea bytes) 2024-05-08T03:19:33.625 INFO:teuthology.orchestra.run.smithi177.stdout:477 read 0x3a96e50 thru 0x3aa5f3d (0xf0ee bytes) 2024-05-08T03:19:33.633 INFO:teuthology.orchestra.run.smithi177.stdout:478 write 0x696f8f2 thru 0x6973dbc (0x44cb bytes) 2024-05-08T03:19:33.643 INFO:teuthology.orchestra.run.smithi177.stdout:481 trunc from 0x6973dbd to 0xc272bf0 2024-05-08T03:19:33.655 INFO:teuthology.orchestra.run.smithi177.stdout:482 trunc from 0xc272bf0 to 0x636ed8b 2024-05-08T03:19:33.668 INFO:teuthology.orchestra.run.smithi177.stdout:483 read 0x3eaa6a1 thru 0x3eb9887 (0xf1e7 bytes) 2024-05-08T03:19:33.669 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:33.677 INFO:teuthology.orchestra.run.smithi177.stdout:486 write 0x89eeb7d thru 0x89fb022 (0xc4a6 bytes) 2024-05-08T03:19:33.687 INFO:teuthology.orchestra.run.smithi177.stdout:487 writesame 0x9d020b9 thru 0x9d11538 data_size 0x18(0xf480 bytes) 2024-05-08T03:19:33.687 INFO:teuthology.orchestra.run.smithi177.stdout:488 clone 6 order 21 su 32768 sc 13 2024-05-08T03:19:33.701 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:33.702 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:33.709 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:33.707+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:33.714 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:33.809 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:33.808+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:33.845 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:19:33.859 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:33.860 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:33.870 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:33.910 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:33.909+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:33.924 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone5 from 0x9d11539 (overlap 0x4c2bca6) to 0x448dcfb 2024-05-08T03:19:34.012 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:34.010+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:34.027 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.027 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.036 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:34.122 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:34.120+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:34.183 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.183 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.192 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:34.232 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:34.231+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:34.315 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:34.313+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:34.337 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.337 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.346 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:34.416 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:34.414+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:34.493 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.493 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:34.517 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:34.516+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:34.607 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.607 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.616 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:34.617 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:34.616+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:34.718 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:34.716+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:34.720 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.721 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.729 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:34.819 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:34.817+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:34.835 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.835 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.844 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:34.848 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:19:34.919 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:34.918+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:34.961 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:34.961 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:34.971 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:34.990 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-05-08T03:19:35.020 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:35.018+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:35.121 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:35.119+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:35.134 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:35.134 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:35.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:35.164 INFO:teuthology.orchestra.run.smithi177.stdout:489 writesame 0x597fe65 thru 0x5986f60 data_size 0xe(0x70fc bytes) 2024-05-08T03:19:35.183 INFO:teuthology.orchestra.run.smithi177.stdout:490 write 0x64b63d7 thru 0x64bb397 (0x4fc1 bytes) 2024-05-08T03:19:35.203 INFO:teuthology.orchestra.run.smithi177.stdout:491 trunc from 0x9d11539 to 0x65dfe61 2024-05-08T03:19:35.222 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:35.221+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:35.238 INFO:teuthology.orchestra.run.smithi177.stdout:492 write 0xf91a33 thru 0xfa0feb (0xf5b9 bytes) 2024-05-08T03:19:35.279 INFO:teuthology.orchestra.run.smithi177.stdout:493 read 0x2fbbb5d thru 0x2fc449e (0x8942 bytes) 2024-05-08T03:19:35.279 INFO:teuthology.orchestra.run.smithi177.stdout:495 read 0x133607b thru 0x133e951 (0x88d7 bytes) 2024-05-08T03:19:35.280 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:35.280 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:35.280 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:35.287 INFO:teuthology.orchestra.run.smithi177.stdout:498 trunc from 0x65dfe61 to 0xbc035af 2024-05-08T03:19:35.289 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:35.292 INFO:teuthology.orchestra.run.smithi177.stdout:499 writesame 0x3ec1bac thru 0x3ec4f66 data_size 0x13(0x33bb bytes) 2024-05-08T03:19:35.309 INFO:teuthology.orchestra.run.smithi177.stdout:501 read 0x9a042ec thru 0x9a08674 (0x4389 bytes) 2024-05-08T03:19:35.309 INFO:teuthology.orchestra.run.smithi177.stdout:502 writesame 0x16be4ab thru 0x16caa85 data_size 0x1(0xc5db bytes) 2024-05-08T03:19:35.332 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:35.333 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:35.331+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:35.345 INFO:teuthology.orchestra.run.smithi177.stdout:503 write 0x2ae5d54 thru 0x2af16ce (0xb97b bytes) 2024-05-08T03:19:35.386 INFO:teuthology.orchestra.run.smithi177.stdout:504 write 0xc187f6a thru 0xc18a7cb (0x2862 bytes) 2024-05-08T03:19:35.392 INFO:teuthology.orchestra.run.smithi177.stdout:505 read 0x268f5a0 thru 0x2690349 (0xdaa bytes) 2024-05-08T03:19:35.393 INFO:teuthology.orchestra.run.smithi177.stdout:506 read 0x212651d thru 0x212c830 (0x6314 bytes) 2024-05-08T03:19:35.394 INFO:teuthology.orchestra.run.smithi177.stdout:508 punch from 0x575302e to 0x575e0ee, (0xb0c0 bytes) 2024-05-08T03:19:35.394 INFO:teuthology.orchestra.run.smithi177.stdout:509 read 0x8461f2e thru 0x846f1cf (0xd2a2 bytes) 2024-05-08T03:19:35.405 INFO:teuthology.orchestra.run.smithi177.stdout:510 trunc from 0xc18a7cc to 0x25008e3 2024-05-08T03:19:35.409 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:35.409 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:35.419 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:35.424 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:35.422+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:35.489 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:35.489 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:35.499 INFO:teuthology.orchestra.run.smithi177.stdout:511 writesame 0xb6ecaab thru 0xb6f3110 data_size 0x33(0x6666 bytes) 2024-05-08T03:19:35.499 INFO:teuthology.orchestra.run.smithi177.stdout:513 writesame 0x64f3597 thru 0x64f5e49 data_size 0x1(0x28b3 bytes) 2024-05-08T03:19:35.505 INFO:teuthology.orchestra.run.smithi177.stdout:514 read 0x683bd22 thru 0x6847f66 (0xc245 bytes) 2024-05-08T03:19:35.505 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:35.508 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:35.509 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:35.513 INFO:teuthology.orchestra.run.smithi177.stdout:517 writesame 0xd0bd5c5 thru 0xd0c0d97 data_size 0x1(0x37d3 bytes) 2024-05-08T03:19:35.513 INFO:teuthology.orchestra.run.smithi177.stdout:518 writesame 0x7b9fb2c thru 0x7ba3d0b data_size 0x22(0x41e0 bytes) 2024-05-08T03:19:35.519 INFO:teuthology.orchestra.run.smithi177.stdout:519 writesame 0x7f9100f thru 0x7f9cb47 data_size 0x7(0xbb39 bytes) 2024-05-08T03:19:35.525 INFO:teuthology.orchestra.run.smithi177.stdout:521 read 0xca120c7 thru 0xca2141c (0xf356 bytes) 2024-05-08T03:19:35.525 INFO:teuthology.orchestra.run.smithi177.stdout:522 read 0xc880a0c thru 0xc88a33e (0x9933 bytes) 2024-05-08T03:19:35.525 INFO:teuthology.orchestra.run.smithi177.stdout:524 punch from 0x896df11 to 0x897b2ba, (0xd3a9 bytes) 2024-05-08T03:19:35.525 INFO:teuthology.orchestra.run.smithi177.stdout:525 write 0x160223d thru 0x1607791 (0x5555 bytes) 2024-05-08T03:19:35.525 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:35.523+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:35.528 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:35.528 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:35.535 INFO:teuthology.orchestra.run.smithi177.stdout:526 read 0x31f80ac thru 0x31f81ec (0x141 bytes) 2024-05-08T03:19:35.535 INFO:teuthology.orchestra.run.smithi177.stdout:527 read 0xdc606b thru 0xdcc7fd (0x6793 bytes) 2024-05-08T03:19:35.536 INFO:teuthology.orchestra.run.smithi177.stdout:528 punch from 0x5da7184 to 0x5db05dc, (0x9458 bytes) 2024-05-08T03:19:35.536 INFO:teuthology.orchestra.run.smithi177.stdout:529 writesame 0xb9bff96 thru 0xb9c733b data_size 0x2(0x73a6 bytes) 2024-05-08T03:19:35.546 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:35.546 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:35.546 INFO:teuthology.orchestra.run.smithi177.stdout:530 read 0x6bd23d2 thru 0x6bd5830 (0x345f bytes) 2024-05-08T03:19:35.546 INFO:teuthology.orchestra.run.smithi177.stdout:531 write 0x1a73543 thru 0x1a7bfda (0x8a98 bytes) 2024-05-08T03:19:35.549 INFO:teuthology.orchestra.run.smithi177.stdout:532 read 0x6951d10 thru 0x695fd90 (0xe081 bytes) 2024-05-08T03:19:35.549 INFO:teuthology.orchestra.run.smithi177.stdout:533 write 0x1f909ee thru 0x1f9ead0 (0xe0e3 bytes) 2024-05-08T03:19:35.554 INFO:teuthology.orchestra.run.smithi177.stdout:534 read 0x678dac0 thru 0x679afd9 (0xd51a bytes) 2024-05-08T03:19:35.554 INFO:teuthology.orchestra.run.smithi177.stdout:535 write 0xba20b9 thru 0xba402e (0x1f76 bytes) 2024-05-08T03:19:35.563 INFO:teuthology.orchestra.run.smithi177.stdout:537 write 0x9434876 thru 0x943d717 (0x8ea2 bytes) 2024-05-08T03:19:35.569 INFO:teuthology.orchestra.run.smithi177.stdout:540 write 0xac016e2 thru 0xac103ef (0xed0e bytes) 2024-05-08T03:19:35.569 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:35.569 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7090716029999999, 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:19:35.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.584 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7087294249999996, 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.585 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:35.590 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.708690593, 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:35.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:35.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:19:35.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7086445599999998, 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:19:35.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:19:35.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7076273679999998, 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.602 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:19:35.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:19:35.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:19:35.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7075911169999998, 2024-05-08T03:19:35.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:19:35.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:19:35.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:19:35.612 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7075638179999997, 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.613 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:35.618 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7075341250000005, 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.7075086099999996, 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.624 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:35.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6973944440000004, 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:19:35.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:35.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:35.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.635 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4450555989999998, 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:35.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4450372680000001, 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:19:35.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:35.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.644 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4450128070000003, 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:19:35.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:19:35.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4449864640000003, 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:19:35.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:19:35.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4449690730000002, 2024-05-08T03:19:35.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:19:35.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:19:35.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4449517920000003, 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.658 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:19:35.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:19:35.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:19:35.661 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4449243550000004, 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.662 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:19:35.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:19:35.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4449063960000004, 2024-05-08T03:19:35.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:19:35.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:19:35.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4448893099999998, 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:19:35.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:19:35.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.4448723259999996, 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:35.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:35.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:35.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:35.679 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:35.680 INFO:teuthology.orchestra.run.smithi177.stdout:541 trunc from 0xd0c0d98 to 0x67f05c0 2024-05-08T03:19:35.680 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:35.681 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:35.681 INFO:teuthology.orchestra.run.smithi177.stdout:543 write 0x1a97f19 thru 0x1aa4304 (0xc3ec bytes) 2024-05-08T03:19:35.681 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:35.682 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:35.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:35.683 INFO:teuthology.orchestra.run.smithi177.stdout:544 read 0x3d0e941 thru 0x3d17c5b (0x931b bytes) 2024-05-08T03:19:35.683 INFO:teuthology.orchestra.run.smithi177.stdout:546 write 0x981e5db thru 0x981f10a (0xb30 bytes) 2024-05-08T03:19:35.683 INFO:teuthology.orchestra.run.smithi177.stdout:550 write 0x4799f84 thru 0x479d125 (0x31a2 bytes) 2024-05-08T03:19:35.683 INFO:teuthology.orchestra.run.smithi177.stdout:551 writesame 0x2d30d54 thru 0x2d39421 data_size 0x3a(0x86ce bytes) 2024-05-08T03:19:35.683 INFO:teuthology.orchestra.run.smithi177.stdout:552 writesame 0x69cc387 thru 0x69d9ab7 data_size 0x3(0xd731 bytes) 2024-05-08T03:19:35.684 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:35.684 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:35.684 INFO:teuthology.orchestra.run.smithi177.stdout:553 trunc from 0x981f10b to 0xe7cb013 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:554 read 0x60cb519 thru 0x60d3d4a (0x8832 bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:555 write 0x95a077f thru 0x95a6f1c (0x679e bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:558 punch from 0xe79778 to 0xe7a53b, (0xdc3 bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:559 write 0x56d56f8 thru 0x56dd973 (0x827c bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:560 write 0x3d4eb2a thru 0x3d51329 (0x2800 bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:565 writesame 0x87bfee3 thru 0x87c5504 data_size 0x4b(0x5622 bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:568 write 0x791a301 thru 0x791a4da (0x1da bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:569 punch from 0xdecbd6a to 0xdecdce9, (0x1f7f bytes) 2024-05-08T03:19:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:570 writesame 0xbb2a76d thru 0xbb2f981 data_size 0x1(0x5215 bytes) 2024-05-08T03:19:35.687 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:35.678+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:35.687 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:35.687 INFO:teuthology.orchestra.run.smithi177.stdout:572 write 0x266de3e thru 0x267b012 (0xd1d5 bytes) 2024-05-08T03:19:35.689 INFO:teuthology.orchestra.run.smithi177.stdout:576 read 0x17bb7b5 thru 0x17c6b01 (0xb34d bytes) 2024-05-08T03:19:35.689 INFO:teuthology.orchestra.run.smithi177.stdout:577 write 0xb4e3031 thru 0xb4e7d0d (0x4cdd bytes) 2024-05-08T03:19:35.693 INFO:teuthology.orchestra.run.smithi177.stdout:579 flatten 2024-05-08T03:19:35.731 INFO:teuthology.orchestra.run.smithi177.stdout:581 punch from 0xbf8c9e0 to 0xbf90b1f, (0x413f bytes) 2024-05-08T03:19:35.731 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:35.731 INFO:teuthology.orchestra.run.smithi177.stdout:583 writesame 0x855c1cc thru 0x855c54c data_size 0x17(0x381 bytes) 2024-05-08T03:19:35.738 INFO:teuthology.orchestra.run.smithi177.stdout:584 write 0xe90f47d thru 0xe915890 (0x6414 bytes) 2024-05-08T03:19:35.743 INFO:teuthology.orchestra.run.smithi177.stdout:585 write 0x93e15dc thru 0x93efcce (0xe6f3 bytes) 2024-05-08T03:19:35.747 INFO:teuthology.orchestra.run.smithi177.stdout:586 read 0x600dc39 thru 0x600f8c7 (0x1c8f bytes) 2024-05-08T03:19:35.747 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:35.747 INFO:teuthology.orchestra.run.smithi177.stdout:589 write 0xb67eb9d thru 0xb68a7f5 (0xbc59 bytes) 2024-05-08T03:19:35.765 INFO:teuthology.orchestra.run.smithi177.stdout:590 trunc from 0xe915891 to 0x43343a9 2024-05-08T03:19:35.781 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:35.780+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:35.788 INFO:teuthology.orchestra.run.smithi177.stdout:591 read 0x39d90e0 thru 0x39df9cb (0x68ec bytes) 2024-05-08T03:19:35.788 INFO:teuthology.orchestra.run.smithi177.stdout:594 clone 7 order 25 su 2097152 sc 6 2024-05-08T03:19:35.833 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:35.833 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:35.833 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:35.833 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:35.839 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:35.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:35.840 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T03:19:35.840 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:19:35.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:35.886 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:35.884+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:35.949 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:35.949 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:35.949 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:35.949 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:35.949 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:35.958 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:35.984 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:35.982+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:35.997 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-05-08T03:19:36.084 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:36.083+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:36.109 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:36.109 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:36.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:36.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4298 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208457+0000", 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8990155660000001, 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019038892000000002, 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4298 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.110 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208457+0000", 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208457+0000", 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208457+0000", 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:36.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208458+0000", 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.35e-06 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208461+0000", 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0170000000000001e-06 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208470+0000", 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6999999999999997e-06 2024-05-08T03:19:36.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227324+0000", 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018853687000000001 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227405+0000", 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0808000000000003e-05 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227496+0000", 2024-05-08T03:19:36.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.09e-05 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4299 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208476+0000", 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8989957500000001, 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019254543999999998, 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4299 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208476+0000", 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208476+0000", 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208477+0000", 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.72e-07 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208479+0000", 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0810000000000001e-06 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208485+0000", 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3099999999999997e-06 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208495+0000", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6860000000000001e-06 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227549+0000", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019054075 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227629+0000", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9658000000000002e-05 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227731+0000", 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010236199999999999 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4301 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208502+0000", 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8989701600000002, 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019425779000000001, 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.118 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4301 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208502+0000", 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208502+0000", 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208502+0000", 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2e-07 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208505+0000", 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1239999999999999e-06 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208511+0000", 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208516+0000", 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0119999999999996e-06 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227766+0000", 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019250587999999999 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227839+0000", 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2911999999999996e-05 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227928+0000", 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8537999999999998e-05 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.122 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4302 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.123 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208522+0000", 2024-05-08T03:19:36.123 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8989503120000002, 2024-05-08T03:19:36.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019643984, 2024-05-08T03:19:36.123 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4302 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208522+0000", 2024-05-08T03:19:36.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208522+0000", 2024-05-08T03:19:36.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208522+0000", 2024-05-08T03:19:36.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:36.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208524+0000", 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9439999999999999e-06 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208530+0000", 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8449999999999999e-06 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208535+0000", 2024-05-08T03:19:36.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8709999999999996e-06 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.227977+0000", 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019442575 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228062+0000", 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4427999999999999e-05 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228166+0000", 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103955 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4304 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208559+0000", 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8989129350000002, 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019790359, 2024-05-08T03:19:36.129 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4304 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208559+0000", 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208559+0000", 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208560+0000", 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.75e-07 2024-05-08T03:19:36.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208566+0000", 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2219999999999999e-06 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208574+0000", 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3319999999999992e-06 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208580+0000", 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:19:36.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228194+0000", 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019613865000000001 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228264+0000", 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0382000000000005e-05 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228350+0000", 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5296999999999993e-05 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.133 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4306 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208588+0000", 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8988845200000002, 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020011376000000001, 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.134 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4306 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208588+0000", 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208588+0000", 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208588+0000", 2024-05-08T03:19:36.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8299999999999997e-07 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208590+0000", 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.018e-06 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208597+0000", 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1690000000000003e-06 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208604+0000", 2024-05-08T03:19:36.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4239999999999998e-06 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228405+0000", 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019801236999999999 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228490+0000", 2024-05-08T03:19:36.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4730999999999999e-05 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228599+0000", 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000109314 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4309 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.138 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208610+0000", 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8988618759999998, 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020167729999999998, 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4309 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208610+0000", 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208610+0000", 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208611+0000", 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2599999999999998e-07 2024-05-08T03:19:36.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208613+0000", 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9690000000000001e-06 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208621+0000", 2024-05-08T03:19:36.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4030000000000007e-06 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208626+0000", 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8879999999999996e-06 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228621+0000", 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019994997 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228692+0000", 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0611000000000005e-05 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228778+0000", 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6435999999999997e-05 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.143 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4310 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208634+0000", 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8988385480000001, 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020379050999999999, 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.144 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4310 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208634+0000", 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208634+0000", 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208634+0000", 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9900000000000001e-07 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208642+0000", 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2360000000000004e-06 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208649+0000", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.674e-06 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208654+0000", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1580000000000004e-06 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228826+0000", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020171962000000002 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.228909+0000", 2024-05-08T03:19:36.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3041999999999995e-05 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229013+0000", 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010357999999999999 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4312 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208662+0000", 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8988101369999999, 2024-05-08T03:19:36.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020547106999999998, 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4312 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208662+0000", 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208662+0000", 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208662+0000", 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0699999999999998e-07 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208665+0000", 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1280000000000002e-06 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208669+0000", 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8629999999999999e-06 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208672+0000", 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8090000000000001e-06 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229048+0000", 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020375642999999999 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229118+0000", 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0018000000000006e-05 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229209+0000", 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1238999999999998e-05 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.152 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4314 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208677+0000", 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8987952020000001, 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020688754, 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.153 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4314 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208677+0000", 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208677+0000", 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208677+0000", 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1400000000000001e-07 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208678+0000", 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1090000000000001e-06 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208681+0000", 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.106e-06 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208688+0000", 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1489999999999999e-06 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229243+0000", 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020555545000000001 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229298+0000", 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5077999999999997e-05 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229366+0000", 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7552999999999995e-05 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4315 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208691+0000", 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8987811739999998, 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020863024000000001, 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.157 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4315 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208691+0000", 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208691+0000", 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208691+0000", 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.11e-07 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208692+0000", 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.119e-06 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208695+0000", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.965e-06 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208698+0000", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3690000000000001e-06 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229386+0000", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020688656999999999 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229461+0000", 2024-05-08T03:19:36.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4795000000000005e-05 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229554+0000", 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2907999999999999e-05 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4317 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 450560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208701+0000", 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.898771268, 2024-05-08T03:19:36.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020981265999999998, 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4317 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208701+0000", 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208701+0000", 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208701+0000", 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2399999999999999e-07 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208702+0000", 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5499999999999996e-07 2024-05-08T03:19:36.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208705+0000", 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9490000000000001e-06 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208707+0000", 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.401e-06 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229582+0000", 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020875013000000001 2024-05-08T03:19:36.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229624+0000", 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1145e-05 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229682+0000", 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8579000000000003e-05 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4319 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 458752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.165 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208710+0000", 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.898761758, 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021169173999999999, 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4319 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208710+0000", 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208710+0000", 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208711+0000", 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1400000000000001e-07 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208715+0000", 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9330000000000003e-06 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208718+0000", 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0620000000000001e-06 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208720+0000", 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2579999999999998e-06 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229717+0000", 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.020997261999999999 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229789+0000", 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1415999999999995e-05 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229880+0000", 2024-05-08T03:19:36.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1028999999999999e-05 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4320 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.171 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208723+0000", 2024-05-08T03:19:36.173 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8987491529999998, 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021293261000000001, 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.174 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4320 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208723+0000", 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208723+0000", 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208723+0000", 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.11e-07 2024-05-08T03:19:36.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208724+0000", 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0210000000000001e-06 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208727+0000", 2024-05-08T03:19:36.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.819e-06 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208729+0000", 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.26e-06 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229908+0000", 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021178375999999999 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.229953+0000", 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5028000000000003e-05 2024-05-08T03:19:36.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230016+0000", 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3546000000000004e-05 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.180 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4321 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208732+0000", 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8987399599999999, 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021474815000000001, 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.181 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4321 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208732+0000", 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208732+0000", 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208732+0000", 2024-05-08T03:19:36.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.11e-07 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208733+0000", 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4e-07 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208736+0000", 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8140000000000001e-06 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208738+0000", 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.255e-06 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230051+0000", 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021312267999999999 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230118+0000", 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7138999999999995e-05 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230207+0000", 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9188000000000001e-05 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4323 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208741+0000", 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8987308349999998, 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021588855000000001, 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.186 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4323 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208741+0000", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208741+0000", 2024-05-08T03:19:36.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208742+0000", 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208743+0000", 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7100000000000011e-07 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208745+0000", 2024-05-08T03:19:36.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8839999999999998e-06 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208748+0000", 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2869999999999999e-06 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230234+0000", 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021486472999999999 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230274+0000", 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0061000000000002e-05 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230330+0000", 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5968999999999999e-05 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4325 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 491520~1078] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208751+0000", 2024-05-08T03:19:36.190 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8987215150000001, 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021830787000000001, 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4325 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208751+0000", 2024-05-08T03:19:36.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208751+0000", 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208751+0000", 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.53e-07 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208757+0000", 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9819999999999996e-06 2024-05-08T03:19:36.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208762+0000", 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0969999999999996e-06 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208764+0000", 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4430000000000002e-06 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230373+0000", 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021608315999999999 2024-05-08T03:19:36.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230468+0000", 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5711000000000006e-05 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230581+0000", 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000112985 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.194 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4326 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208781+0000", 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.898691189, 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021924282999999999, 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4326 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208781+0000", 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208781+0000", 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208782+0000", 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.196 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208784+0000", 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.215e-06 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208790+0000", 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0589999999999999e-06 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208795+0000", 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.685e-06 2024-05-08T03:19:36.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230616+0000", 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021821803000000001 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230660+0000", 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3455000000000003e-05 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230705+0000", 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5476999999999998e-05 2024-05-08T03:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4328 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208801+0000", 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8986709149999998, 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022083794, 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.199 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4328 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208801+0000", 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208801+0000", 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208802+0000", 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5600000000000001e-07 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208812+0000", 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0063e-05 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208820+0000", 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2139999999999996e-06 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208834+0000", 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3969e-05 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230741+0000", 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021906913 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230816+0000", 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4857999999999994e-05 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230885+0000", 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9420999999999998e-05 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:4331 3.16 3:6d9d8170:::rbd_data.11821677beed.0000000000000003:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.203 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.208841+0000", 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 2.8986307350000002, 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022152053000000001, 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 4331 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208841+0000", 2024-05-08T03:19:36.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208841+0000", 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208842+0000", 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.58e-07 2024-05-08T03:19:36.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208844+0000", 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2910000000000002e-06 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208852+0000", 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.208861+0000", 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9369999999999993e-06 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230902+0000", 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022040635999999999 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230951+0000", 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8856000000000001e-05 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.230993+0000", 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2969999999999997e-05 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.208 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:595 punch from 0x42c7c9f to 0x42caee3, (0x3244 bytes) 2024-05-08T03:19:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:596 write 0xaee06bd thru 0xaee607e (0x59c2 bytes) 2024-05-08T03:19:36.210 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 33, 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:19:35.844298+0000", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:19:10.716373+0000", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:19:30.813350+0000", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5806080, 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "nodeep-scrub", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "noscrub", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:19:36.211 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 5, 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "jewel", 2024-05-08T03:19:36.212 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:19:36.213 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:19:36.214 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:19:36.215 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:19:36.216 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:19:36.217 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 1.9900000095367432, 2024-05-08T03:19:36.218 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 1.9900000095367432, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.87999999523162842, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.75, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.75, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 1, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 1 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:19:36.219 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:19:36.220 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:19:36.221 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:19:36.222 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:19:36.223 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:19:36.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 8, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 8, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.25, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 1, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 1 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:19:36.226 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:19:36.227 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "33", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 9, 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 33, 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:19:36.228 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:19:36.229 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:19:36.230 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 1.7400000095367432, 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 1.7400000095367432, 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.87999999523162842, 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.5299999713897705, 2024-05-08T03:19:36.231 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.5299999713897705, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 1, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 1 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.232 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 28, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.237 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 28, 2024-05-08T03:19:36.238 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:19:36.239 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.240 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.241 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.242 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:19:36.243 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.244 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 28, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.245 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.248 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:19:36.249 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.250 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 19, 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.251 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:19:36.252 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:19:36.253 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:19:36.255 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.256 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 19, 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.261 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.262 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 28, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.266 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1643429991", 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1643429991", 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1643429991", 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1643429991", 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 28, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.270 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.271 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.272 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:19:36.273 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:19:36.274 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:19:36.275 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.277 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.278 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:19:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:19:36.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:19:36.282 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:19:36.283 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:19:36.284 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.285 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.287 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.288 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:19:36.289 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [], 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [], 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:19:36.290 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [], 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [], 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:19:36.291 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:19:36.292 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:19:36.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:36.292 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:36.292 INFO:teuthology.orchestra.run.smithi177.stdout:597 writesame 0x896f48a thru 0x897e1dd data_size 0xb7(0xed54 bytes) 2024-05-08T03:19:36.292 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:36.294 INFO:tasks.thrashosds.thrasher:j is {'epoch': 33, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:19:35.844298+0000', 'last_up_change': '2024-05-08T03:19:10.716373+0000', 'last_in_change': '2024-05-08T03:19:30.813350+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 33, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1643429991}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1643429991}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1643429991}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1643429991}]}, 'public_addr': '172.21.15.188:6817/1643429991', 'cluster_addr': '172.21.15.188:6819/1643429991', 'heartbeat_back_addr': '172.21.15.188:6823/1643429991', 'heartbeat_front_addr': '172.21.15.188:6821/1643429991', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:19:36.294 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:19:36.295 INFO:teuthology.orchestra.run.smithi177.stdout:598 read 0x52bea96 thru 0x52c4de6 (0x6351 bytes) 2024-05-08T03:19:36.295 INFO:teuthology.orchestra.run.smithi177.stdout:599 punch from 0x86c9e37 to 0x86d98cd, (0xfa96 bytes) 2024-05-08T03:19:36.295 INFO:teuthology.orchestra.run.smithi177.stdout:600 read 0x2e0d88e thru 0x2e1d1ff (0xf972 bytes) 2024-05-08T03:19:36.295 INFO:teuthology.orchestra.run.smithi177.stdout:601 trunc from 0xaee607f to 0x504369a 2024-05-08T03:19:36.296 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:36.208+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:36.297 INFO:teuthology.orchestra.run.smithi177.stdout:603 punch from 0x1c021e5 to 0x1c0aead, (0x8cc8 bytes) 2024-05-08T03:19:36.297 INFO:teuthology.orchestra.run.smithi177.stdout:604 punch from 0x157f7d to 0x15a064, (0x20e7 bytes) 2024-05-08T03:19:36.297 INFO:teuthology.orchestra.run.smithi177.stdout:605 write 0xeceab8b thru 0xecf2ebf (0x8335 bytes) 2024-05-08T03:19:36.297 INFO:teuthology.orchestra.run.smithi177.stdout:606 trunc from 0xecf2ec0 to 0x9e0caa8 2024-05-08T03:19:36.297 INFO:teuthology.orchestra.run.smithi177.stdout:607 clone 8 order 21 su 131072 sc 12 2024-05-08T03:19:36.334 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:36.333+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:36.400 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:36.400 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:36.401 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:36.401 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:36.409 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:36.411 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:36.409+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:36.512 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:36.510+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:36.514 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:36.514 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:36.514 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:36.514 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:36.514 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:36.524 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:36.613 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:36.611+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:36.617 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:19:36.618 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:19:36.629 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-05-08T03:19:36.177464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":50783918,"num_objects":118,"num_object_clones":27,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":43,"num_read":6166,"num_read_kb":8142,"num_write":1262,"num_write_kb":6797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1044,"ondisk_log_size":1044,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":206784,"kb_used_data":5156,"kb_used_omap":10,"kb_used_meta":201525,"kb_avail":660396096,"statfs":{"total":676457349120,"available":676245602304,"internally_reserved":0,"allocated":5279744,"data_stored":3716811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":206361707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000634"},"pg_stats":[{"pgid":"3.1f","version":"28'13","reported_seq":53,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.823742+0000","last_change":"2024-05-08T03:19:22.791406+0000","last_active":"2024-05-08T03:19:30.823742+0000","last_peered":"2024-05-08T03:19:30.823742+0000","last_clean":"2024-05-08T03:19:30.823742+0000","last_became_active":"2024-05-08T03:19:20.776736+0000","last_became_peered":"2024-05-08T03:19:20.776736+0000","last_unstale":"2024-05-08T03:19:30.823742+0000","last_undegraded":"2024-05-08T03:19:30.823742+0000","last_fullsized":"2024-05-08T03:19:30.823742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:05.749826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":45129,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":13,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'17","reported_seq":61,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.826437+0000","last_change":"2024-05-08T03:19:22.850001+0000","last_active":"2024-05-08T03:19:30.826437+0000","last_peered":"2024-05-08T03:19:30.826437+0000","last_clean":"2024-05-08T03:19:30.826437+0000","last_became_active":"2024-05-08T03:19:20.773128+0000","last_became_peered":"2024-05-08T03:19:20.773128+0000","last_unstale":"2024-05-08T03:19:30.826437+0000","last_undegraded":"2024-05-08T03:19:30.826437+0000","last_fullsized":"2024-05-08T03:19:30.826437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:22.078675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":4272861,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":14,"num_read_kb":3,"num_write":15,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"27'21","reported_seq":60,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.966900+0000","last_change":"2024-05-08T03:19:30.828828+0000","last_active":"2024-05-08T03:19:30.828828+0000","last_peered":"2024-05-08T03:19:29.966900+0000","last_clean":"2024-05-08T03:19:29.966900+0000","last_became_active":"2024-05-08T03:19:20.778943+0000","last_became_peered":"2024-05-08T03:19:20.778943+0000","last_unstale":"2024-05-08T03:19:29.966900+0000","last_undegraded":"2024-05-08T03:19:29.966900+0000","last_fullsized":"2024-05-08T03:19:29.966900+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":41,"num_read_kb":28,"num_write":25,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"27'17","reported_seq":82,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.994340+0000","last_change":"2024-05-08T03:19:30.828944+0000","last_active":"2024-05-08T03:19:30.828944+0000","last_peered":"2024-05-08T03:19:29.994340+0000","last_clean":"2024-05-08T03:19:29.994340+0000","last_became_active":"2024-05-08T03:19:20.776076+0000","last_became_peered":"2024-05-08T03:19:20.776076+0000","last_unstale":"2024-05-08T03:19:29.994340+0000","last_undegraded":"2024-05-08T03:19:29.994340+0000","last_fullsized":"2024-05-08T03:19:29.994340+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":2314338,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":39,"num_read_kb":60,"num_write":14,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":423,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:30.001874+0000","last_change":"2024-05-08T03:19:30.825161+0000","last_active":"2024-05-08T03:19:30.825161+0000","last_peered":"2024-05-08T03:19:30.001874+0000","last_clean":"2024-05-08T03:19:30.001874+0000","last_became_active":"2024-05-08T03:19:20.779326+0000","last_became_peered":"2024-05-08T03:19:20.779326+0000","last_unstale":"2024-05-08T03:19:30.001874+0000","last_undegraded":"2024-05-08T03:19:30.001874+0000","last_fullsized":"2024-05-08T03:19:30.001874+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":3471665,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":605,"num_read_kb":1425,"num_write":17,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"27'31","reported_seq":538,"reported_epoch":28,"state":"peering","last_fresh":"2024-05-08T03:19:30.820711+0000","last_change":"2024-05-08T03:19:30.820711+0000","last_active":"2024-05-08T03:19:30.820711+0000","last_peered":"2024-05-08T03:19:29.885338+0000","last_clean":"2024-05-08T03:19:29.885338+0000","last_became_active":"2024-05-08T03:19:20.775915+0000","last_became_peered":"2024-05-08T03:19:20.775915+0000","last_unstale":"2024-05-08T03:19:30.820711+0000","last_undegraded":"2024-05-08T03:19:30.820711+0000","last_fullsized":"2024-05-08T03:19:30.820711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":4613106,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":261,"num_read_kb":171,"num_write":39,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"27'18","reported_seq":59,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.821206+0000","last_change":"2024-05-08T03:19:22.779129+0000","last_active":"2024-05-08T03:19:30.821206+0000","last_peered":"2024-05-08T03:19:30.821206+0000","last_clean":"2024-05-08T03:19:30.821206+0000","last_became_active":"2024-05-08T03:19:20.775300+0000","last_became_peered":"2024-05-08T03:19:20.775300+0000","last_unstale":"2024-05-08T03:19:30.821206+0000","last_undegraded":"2024-05-08T03:19:30.821206+0000","last_fullsized":"2024-05-08T03:19:30.821206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:24:18.637967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":624470,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":9,"num_read_kb":68,"num_write":16,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"27'76","reported_seq":885,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.825976+0000","last_change":"2024-05-08T03:19:22.849651+0000","last_active":"2024-05-08T03:19:30.825976+0000","last_peered":"2024-05-08T03:19:30.825976+0000","last_clean":"2024-05-08T03:19:30.825976+0000","last_became_active":"2024-05-08T03:19:20.774922+0000","last_became_peered":"2024-05-08T03:19:20.774922+0000","last_unstale":"2024-05-08T03:19:30.825976+0000","last_undegraded":"2024-05-08T03:19:30.825976+0000","last_fullsized":"2024-05-08T03:19:30.825976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:38.871931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":5530062,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":1444,"num_read_kb":2532,"num_write":72,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"27'17","reported_seq":53,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.823975+0000","last_change":"2024-05-08T03:19:22.791865+0000","last_active":"2024-05-08T03:19:30.823975+0000","last_peered":"2024-05-08T03:19:30.823975+0000","last_clean":"2024-05-08T03:19:30.823975+0000","last_became_active":"2024-05-08T03:19:20.776384+0000","last_became_peered":"2024-05-08T03:19:20.776384+0000","last_unstale":"2024-05-08T03:19:30.823975+0000","last_undegraded":"2024-05-08T03:19:30.823975+0000","last_fullsized":"2024-05-08T03:19:30.823975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:43.213262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":3555063,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":2,"num_read_kb":104,"num_write":15,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"27'13","reported_seq":180,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.828628+0000","last_change":"2024-05-08T03:19:22.787101+0000","last_active":"2024-05-08T03:19:30.828628+0000","last_peered":"2024-05-08T03:19:30.828628+0000","last_clean":"2024-05-08T03:19:30.828628+0000","last_became_active":"2024-05-08T03:19:20.776739+0000","last_became_peered":"2024-05-08T03:19:20.776739+0000","last_unstale":"2024-05-08T03:19:30.828628+0000","last_undegraded":"2024-05-08T03:19:30.828628+0000","last_fullsized":"2024-05-08T03:19:30.828628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:38:12.696362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":734262,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":135,"num_read_kb":54,"num_write":11,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'16","reported_seq":59,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.826451+0000","last_change":"2024-05-08T03:19:22.778584+0000","last_active":"2024-05-08T03:19:30.826451+0000","last_peered":"2024-05-08T03:19:30.826451+0000","last_clean":"2024-05-08T03:19:30.826451+0000","last_became_active":"2024-05-08T03:19:20.780066+0000","last_became_peered":"2024-05-08T03:19:20.780066+0000","last_unstale":"2024-05-08T03:19:30.826451+0000","last_undegraded":"2024-05-08T03:19:30.826451+0000","last_fullsized":"2024-05-08T03:19:30.826451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:52:40.140550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":4797143,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":14,"num_read_kb":4,"num_write":14,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"27'239","reported_seq":273,"reported_epoch":28,"state":"unknown","last_fresh":"2024-05-08T03:19:30.827497+0000","last_change":"2024-05-08T03:19:31.190197+0000","last_active":"2024-05-08T03:19:30.827497+0000","last_peered":"2024-05-08T03:19:29.932592+0000","last_clean":"2024-05-08T03:19:29.932592+0000","last_became_active":"2024-05-08T03:19:20.777605+0000","last_became_peered":"2024-05-08T03:19:20.777605+0000","last_unstale":"2024-05-08T03:19:30.827497+0000","last_undegraded":"2024-05-08T03:19:30.827497+0000","last_fullsized":"2024-05-08T03:19:30.827497+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":296027,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":1203,"num_read_kb":975,"num_write":411,"num_write_kb":494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.822679+0000","last_change":"2024-05-08T03:19:12.731926+0000","last_active":"2024-05-08T03:19:30.822679+0000","last_peered":"2024-05-08T03:19:30.822679+0000","last_clean":"2024-05-08T03:19:30.822679+0000","last_became_active":"2024-05-08T03:19:12.731504+0000","last_became_peered":"2024-05-08T03:19:12.731504+0000","last_unstale":"2024-05-08T03:19:30.822679+0000","last_undegraded":"2024-05-08T03:19:30.822679+0000","last_fullsized":"2024-05-08T03:19:30.822679+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:06:07.983173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"27'20","reported_seq":61,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.829115+0000","last_change":"2024-05-08T03:19:22.787349+0000","last_active":"2024-05-08T03:19:30.829115+0000","last_peered":"2024-05-08T03:19:30.829115+0000","last_clean":"2024-05-08T03:19:30.829115+0000","last_became_active":"2024-05-08T03:19:20.775144+0000","last_became_peered":"2024-05-08T03:19:20.775144+0000","last_unstale":"2024-05-08T03:19:30.829115+0000","last_undegraded":"2024-05-08T03:19:30.829115+0000","last_fullsized":"2024-05-08T03:19:30.829115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:21:04.862004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":1519105,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":14,"num_read_kb":125,"num_write":16,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.823048+0000","last_change":"2024-05-08T03:19:12.966376+0000","last_active":"2024-05-08T03:19:30.823048+0000","last_peered":"2024-05-08T03:19:30.823048+0000","last_clean":"2024-05-08T03:19:30.823048+0000","last_became_active":"2024-05-08T03:19:12.966006+0000","last_became_peered":"2024-05-08T03:19:12.966006+0000","last_unstale":"2024-05-08T03:19:30.823048+0000","last_undegraded":"2024-05-08T03:19:30.823048+0000","last_fullsized":"2024-05-08T03:19:30.823048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:19.151925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"27'21","reported_seq":56,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.821328+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:19:30.821328+0000","last_peered":"2024-05-08T03:19:30.821328+0000","last_clean":"2024-05-08T03:19:30.821328+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:19:30.821328+0000","last_undegraded":"2024-05-08T03:19:30.821328+0000","last_fullsized":"2024-05-08T03:19:30.821328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:01:01.147603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":155648,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":3,"num_read_kb":0,"num_write":19,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":3,"reported_epoch":28,"state":"peering","last_fresh":"2024-05-08T03:19:30.820923+0000","last_change":"2024-05-08T03:19:30.820923+0000","last_active":"2024-05-08T03:19:11.717900+0000","last_peered":"2024-05-08T03:19:11.717900+0000","last_clean":"2024-05-08T03:19:11.717900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T03:19:30.820923+0000","last_undegraded":"2024-05-08T03:19:30.820923+0000","last_fullsized":"2024-05-08T03:19:30.820923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'12","reported_seq":177,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.826475+0000","last_change":"2024-05-08T03:19:22.849986+0000","last_active":"2024-05-08T03:19:30.826475+0000","last_peered":"2024-05-08T03:19:30.826475+0000","last_clean":"2024-05-08T03:19:30.826475+0000","last_became_active":"2024-05-08T03:19:20.777433+0000","last_became_peered":"2024-05-08T03:19:20.777433+0000","last_unstale":"2024-05-08T03:19:30.826475+0000","last_undegraded":"2024-05-08T03:19:30.826475+0000","last_fullsized":"2024-05-08T03:19:30.826475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:43:29.741712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":1374513,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":131,"num_read_kb":18,"num_write":10,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.826252+0000","last_change":"2024-05-08T03:19:12.732433+0000","last_active":"2024-05-08T03:19:30.826252+0000","last_peered":"2024-05-08T03:19:30.826252+0000","last_clean":"2024-05-08T03:19:30.826252+0000","last_became_active":"2024-05-08T03:19:12.732130+0000","last_became_peered":"2024-05-08T03:19:12.732130+0000","last_unstale":"2024-05-08T03:19:30.826252+0000","last_undegraded":"2024-05-08T03:19:30.826252+0000","last_fullsized":"2024-05-08T03:19:30.826252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:37.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"27'20","reported_seq":68,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.823760+0000","last_change":"2024-05-08T03:19:22.791670+0000","last_active":"2024-05-08T03:19:30.823760+0000","last_peered":"2024-05-08T03:19:30.823760+0000","last_clean":"2024-05-08T03:19:30.823760+0000","last_became_active":"2024-05-08T03:19:20.778381+0000","last_became_peered":"2024-05-08T03:19:20.778381+0000","last_unstale":"2024-05-08T03:19:30.823760+0000","last_undegraded":"2024-05-08T03:19:30.823760+0000","last_fullsized":"2024-05-08T03:19:30.823760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:02:08.718403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":1915048,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":17,"num_read_kb":70,"num_write":17,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"27'16","reported_seq":53,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.993842+0000","last_change":"2024-05-08T03:19:30.823992+0000","last_active":"2024-05-08T03:19:30.823992+0000","last_peered":"2024-05-08T03:19:29.993842+0000","last_clean":"2024-05-08T03:19:29.993842+0000","last_became_active":"2024-05-08T03:19:20.781222+0000","last_became_peered":"2024-05-08T03:19:20.781222+0000","last_unstale":"2024-05-08T03:19:29.993842+0000","last_undegraded":"2024-05-08T03:19:29.993842+0000","last_fullsized":"2024-05-08T03:19:29.993842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":4,"num_read_kb":0,"num_write":14,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.825319+0000","last_change":"2024-05-08T03:19:13.735020+0000","last_active":"2024-05-08T03:19:30.825319+0000","last_peered":"2024-05-08T03:19:30.825319+0000","last_clean":"2024-05-08T03:19:30.825319+0000","last_became_active":"2024-05-08T03:19:13.734713+0000","last_became_peered":"2024-05-08T03:19:13.734713+0000","last_unstale":"2024-05-08T03:19:30.825319+0000","last_undegraded":"2024-05-08T03:19:30.825319+0000","last_fullsized":"2024-05-08T03:19:30.825319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:54.753213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.820989+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:19:30.820989+0000","last_peered":"2024-05-08T03:19:30.820989+0000","last_clean":"2024-05-08T03:19:30.820989+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:19:30.820989+0000","last_undegraded":"2024-05-08T03:19:30.820989+0000","last_fullsized":"2024-05-08T03:19:30.820989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"25'5","reported_seq":48,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.820861+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:19:30.820861+0000","last_peered":"2024-05-08T03:19:30.820861+0000","last_clean":"2024-05-08T03:19:30.820861+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:19:30.820861+0000","last_undegraded":"2024-05-08T03:19:30.820861+0000","last_fullsized":"2024-05-08T03:19:30.820861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:53.691047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.825543+0000","last_change":"2024-05-08T03:19:12.966511+0000","last_active":"2024-05-08T03:19:30.825543+0000","last_peered":"2024-05-08T03:19:30.825543+0000","last_clean":"2024-05-08T03:19:30.825543+0000","last_became_active":"2024-05-08T03:19:12.965999+0000","last_became_peered":"2024-05-08T03:19:12.965999+0000","last_unstale":"2024-05-08T03:19:30.825543+0000","last_undegraded":"2024-05-08T03:19:30.825543+0000","last_fullsized":"2024-05-08T03:19:30.825543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:11.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"27'42","reported_seq":257,"reported_epoch":28,"state":"peering","last_fresh":"2024-05-08T03:19:30.824753+0000","last_change":"2024-05-08T03:19:30.824753+0000","last_active":"2024-05-08T03:19:30.824753+0000","last_peered":"2024-05-08T03:19:29.890063+0000","last_clean":"2024-05-08T03:19:29.890063+0000","last_became_active":"2024-05-08T03:19:20.778774+0000","last_became_peered":"2024-05-08T03:19:20.778774+0000","last_unstale":"2024-05-08T03:19:30.824753+0000","last_undegraded":"2024-05-08T03:19:30.824753+0000","last_fullsized":"2024-05-08T03:19:30.824753+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":179277,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":546,"num_read_kb":472,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.828455+0000","last_change":"2024-05-08T03:19:12.965962+0000","last_active":"2024-05-08T03:19:30.828455+0000","last_peered":"2024-05-08T03:19:30.828455+0000","last_clean":"2024-05-08T03:19:30.828455+0000","last_became_active":"2024-05-08T03:19:12.965629+0000","last_became_peered":"2024-05-08T03:19:12.965629+0000","last_unstale":"2024-05-08T03:19:30.828455+0000","last_undegraded":"2024-05-08T03:19:30.828455+0000","last_fullsized":"2024-05-08T03:19:30.828455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:50.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"28'53","reported_seq":98,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.831835+0000","last_change":"2024-05-08T03:19:22.850236+0000","last_active":"2024-05-08T03:19:30.831835+0000","last_peered":"2024-05-08T03:19:30.831835+0000","last_clean":"2024-05-08T03:19:30.831835+0000","last_became_active":"2024-05-08T03:19:20.774210+0000","last_became_peered":"2024-05-08T03:19:20.774210+0000","last_unstale":"2024-05-08T03:19:30.831835+0000","last_undegraded":"2024-05-08T03:19:30.831835+0000","last_fullsized":"2024-05-08T03:19:30.831835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:10:12.533968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":61550,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":188,"num_read_kb":141,"num_write":73,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.825449+0000","last_change":"2024-05-08T03:19:12.966516+0000","last_active":"2024-05-08T03:19:30.825449+0000","last_peered":"2024-05-08T03:19:30.825449+0000","last_clean":"2024-05-08T03:19:30.825449+0000","last_became_active":"2024-05-08T03:19:12.966041+0000","last_became_peered":"2024-05-08T03:19:12.966041+0000","last_unstale":"2024-05-08T03:19:30.825449+0000","last_undegraded":"2024-05-08T03:19:30.825449+0000","last_fullsized":"2024-05-08T03:19:30.825449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:08.473548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"27'12","reported_seq":53,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.826135+0000","last_change":"2024-05-08T03:19:22.850234+0000","last_active":"2024-05-08T03:19:30.826135+0000","last_peered":"2024-05-08T03:19:30.826135+0000","last_clean":"2024-05-08T03:19:30.826135+0000","last_became_active":"2024-05-08T03:19:20.773661+0000","last_became_peered":"2024-05-08T03:19:20.773661+0000","last_unstale":"2024-05-08T03:19:30.826135+0000","last_undegraded":"2024-05-08T03:19:30.826135+0000","last_fullsized":"2024-05-08T03:19:30.826135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:03:22.450592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":16,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":11,"num_read_kb":2,"num_write":10,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"25'6","reported_seq":35,"reported_epoch":28,"state":"peering","last_fresh":"2024-05-08T03:19:30.820603+0000","last_change":"2024-05-08T03:19:30.820603+0000","last_active":"2024-05-08T03:19:30.820603+0000","last_peered":"2024-05-08T03:19:28.236670+0000","last_clean":"2024-05-08T03:19:28.236670+0000","last_became_active":"2024-05-08T03:19:20.777503+0000","last_became_peered":"2024-05-08T03:19:20.777503+0000","last_unstale":"2024-05-08T03:19:30.820603+0000","last_undegraded":"2024-05-08T03:19:30.820603+0000","last_fullsized":"2024-05-08T03:19:30.820603+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":47,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.899587+0000","last_change":"2024-05-08T03:19:30.823717+0000","last_active":"2024-05-08T03:19:30.823717+0000","last_peered":"2024-05-08T03:19:29.899587+0000","last_clean":"2024-05-08T03:19:29.899587+0000","last_became_active":"2024-05-08T03:19:20.779555+0000","last_became_peered":"2024-05-08T03:19:20.779555+0000","last_unstale":"2024-05-08T03:19:29.899587+0000","last_undegraded":"2024-05-08T03:19:29.899587+0000","last_fullsized":"2024-05-08T03:19:29.899587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":262144,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":5,"num_read_kb":5,"num_write":13,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"27'10","reported_seq":46,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.826069+0000","last_change":"2024-05-08T03:19:22.849690+0000","last_active":"2024-05-08T03:19:30.826069+0000","last_peered":"2024-05-08T03:19:30.826069+0000","last_clean":"2024-05-08T03:19:30.826069+0000","last_became_active":"2024-05-08T03:19:20.778699+0000","last_became_peered":"2024-05-08T03:19:20.778699+0000","last_unstale":"2024-05-08T03:19:30.826069+0000","last_undegraded":"2024-05-08T03:19:30.826069+0000","last_fullsized":"2024-05-08T03:19:30.826069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:12:29.369697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1,"num_read_kb":0,"num_write":9,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"27'10","reported_seq":44,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.828753+0000","last_change":"2024-05-08T03:19:22.786564+0000","last_active":"2024-05-08T03:19:30.828753+0000","last_peered":"2024-05-08T03:19:30.828753+0000","last_clean":"2024-05-08T03:19:30.828753+0000","last_became_active":"2024-05-08T03:19:20.775160+0000","last_became_peered":"2024-05-08T03:19:20.775160+0000","last_unstale":"2024-05-08T03:19:30.828753+0000","last_undegraded":"2024-05-08T03:19:30.828753+0000","last_fullsized":"2024-05-08T03:19:30.828753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:37.374985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":181611,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"28'27","reported_seq":357,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.838296+0000","last_change":"2024-05-08T03:19:22.778638+0000","last_active":"2024-05-08T03:19:30.838296+0000","last_peered":"2024-05-08T03:19:30.838296+0000","last_clean":"2024-05-08T03:19:30.838296+0000","last_became_active":"2024-05-08T03:19:20.777679+0000","last_became_peered":"2024-05-08T03:19:20.777679+0000","last_unstale":"2024-05-08T03:19:30.838296+0000","last_undegraded":"2024-05-08T03:19:30.838296+0000","last_fullsized":"2024-05-08T03:19:30.838296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:33:08.656143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":5465018,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":354,"num_read_kb":282,"num_write":27,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"27'69","reported_seq":128,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.905654+0000","last_change":"2024-05-08T03:19:30.826895+0000","last_active":"2024-05-08T03:19:30.826895+0000","last_peered":"2024-05-08T03:19:29.905654+0000","last_clean":"2024-05-08T03:19:29.905654+0000","last_became_active":"2024-05-08T03:19:20.779978+0000","last_became_peered":"2024-05-08T03:19:20.779978+0000","last_unstale":"2024-05-08T03:19:29.905654+0000","last_undegraded":"2024-05-08T03:19:29.905654+0000","last_fullsized":"2024-05-08T03:19:29.905654+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":3737460,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":249,"num_read_kb":344,"num_write":94,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.865965+0000","last_change":"2024-05-08T03:19:30.826238+0000","last_active":"2024-05-08T03:19:30.826238+0000","last_peered":"2024-05-08T03:19:29.865965+0000","last_clean":"2024-05-08T03:19:29.865965+0000","last_became_active":"2024-05-08T03:19:20.780407+0000","last_became_peered":"2024-05-08T03:19:20.780407+0000","last_unstale":"2024-05-08T03:19:29.865965+0000","last_undegraded":"2024-05-08T03:19:29.865965+0000","last_fullsized":"2024-05-08T03:19:29.865965+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"27'10","reported_seq":53,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.822627+0000","last_change":"2024-05-08T03:19:22.848968+0000","last_active":"2024-05-08T03:19:30.822627+0000","last_peered":"2024-05-08T03:19:30.822627+0000","last_clean":"2024-05-08T03:19:30.822627+0000","last_became_active":"2024-05-08T03:19:20.776537+0000","last_became_peered":"2024-05-08T03:19:20.776537+0000","last_unstale":"2024-05-08T03:19:30.822627+0000","last_undegraded":"2024-05-08T03:19:30.822627+0000","last_fullsized":"2024-05-08T03:19:30.822627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:38.526454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":262181,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":16,"num_read_kb":8,"num_write":9,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"27'15","reported_seq":48,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.903049+0000","last_change":"2024-05-08T03:19:30.825840+0000","last_active":"2024-05-08T03:19:30.825840+0000","last_peered":"2024-05-08T03:19:29.903049+0000","last_clean":"2024-05-08T03:19:29.903049+0000","last_became_active":"2024-05-08T03:19:20.779578+0000","last_became_peered":"2024-05-08T03:19:20.779578+0000","last_unstale":"2024-05-08T03:19:29.903049+0000","last_undegraded":"2024-05-08T03:19:29.903049+0000","last_fullsized":"2024-05-08T03:19:29.903049+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":111200,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'34","reported_seq":390,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-05-08T03:19:30.823647+0000","last_change":"2024-05-08T03:19:22.791411+0000","last_active":"2024-05-08T03:19:30.823647+0000","last_peered":"2024-05-08T03:19:30.823647+0000","last_clean":"2024-05-08T03:19:30.823647+0000","last_became_active":"2024-05-08T03:19:20.777475+0000","last_became_peered":"2024-05-08T03:19:20.777475+0000","last_unstale":"2024-05-08T03:19:30.823647+0000","last_undegraded":"2024-05-08T03:19:30.823647+0000","last_fullsized":"2024-05-08T03:19:30.823647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:35.025615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":714,"num_read_kb":1146,"num_write":33,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"27'24","reported_seq":55,"reported_epoch":27,"state":"peering","last_fresh":"2024-05-08T03:19:29.903087+0000","last_change":"2024-05-08T03:19:30.826137+0000","last_active":"2024-05-08T03:19:30.826137+0000","last_peered":"2024-05-08T03:19:29.903087+0000","last_clean":"2024-05-08T03:19:29.903087+0000","last_became_active":"2024-05-08T03:19:20.779472+0000","last_became_peered":"2024-05-08T03:19:20.779472+0000","last_unstale":"2024-05-08T03:19:29.903087+0000","last_undegraded":"2024-05-08T03:19:29.903087+0000","last_fullsized":"2024-05-08T03:19:29.903087+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":520296,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":50193550,"num_objects":116,"num_object_clones":27,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":6050,"num_read_kb":8042,"num_write":1103,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2588672,"data_stored":2216072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":927,"ondisk_log_size":927,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":1188665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":176142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":574128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542152,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":606208,"data_stored":362791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":315392,"data_stored":129913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1093632,"data_stored":880394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":404778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":293689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":172417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":12304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":245182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":496740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":456519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":480688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:19:36.632 INFO:tasks.thrashosds.thrasher:Setting 3.d to [3, 6] 2024-05-08T03:19:36.632 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.d', '3', '6'] 2024-05-08T03:19:36.632 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.d 3 6 2024-05-08T03:19:36.670 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:36.671 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:36.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:36.671 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:36.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.671 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3878 3.10 3:081ca275:::rbd_data.1182d376acdd.000000000000002c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.671 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.030775+0000", 2024-05-08T03:19:36.672 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.6368256699999999, 2024-05-08T03:19:36.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0060171230000000001, 2024-05-08T03:19:36.672 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.672 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3878 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030775+0000", 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030775+0000", 2024-05-08T03:19:36.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030777+0000", 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0979999999999999e-06 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030782+0000", 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3349999999999997e-06 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030793+0000", 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1457999999999999e-05 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030799+0000", 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.631e-06 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030855+0000", 2024-05-08T03:19:36.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6427e-05 2024-05-08T03:19:36.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.030990+0000", 2024-05-08T03:19:36.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000134275 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.036793+0000", 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058028990000000003 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.679 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.680 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.680 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3896 3.10 3:081ca275:::rbd_data.1182d376acdd.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.680 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.037469+0000", 2024-05-08T03:19:36.680 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.6301324830000001, 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0055081619999999996, 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3896 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037469+0000", 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037469+0000", 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037470+0000", 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0449999999999999e-06 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037474+0000", 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6990000000000002e-06 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037483+0000", 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9060000000000008e-06 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037489+0000", 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1519999999999997e-06 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037573+0000", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3922000000000001e-05 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037667+0000", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3893000000000004e-05 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.037789+0000", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000121311 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:36.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.038990+0000", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00120107 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.042826+0000", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038359219999999999 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.042935+0000", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010922899999999999 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.042977+0000", 2024-05-08T03:19:36.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2012999999999999e-05 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3900 3.10 3:0821a895:::rbd_data.1182d376acdd.0000000000000033:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.039540+0000", 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.6280614629999999, 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063987649999999998, 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.686 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3900 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039540+0000", 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039540+0000", 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039540+0000", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2999999999999999e-07 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039545+0000", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6310000000000002e-06 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039557+0000", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1935e-05 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039566+0000", 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0319999999999992e-06 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039618+0000", 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1557999999999997e-05 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.039725+0000", 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010707799999999999 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.045938+0000", 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062137010000000003 2024-05-08T03:19:36.690 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3926 3.10 3:0fff1983:::rbd_data.1182d376acdd.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.049421+0000", 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.618179832, 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0052598829999999999, 2024-05-08T03:19:36.691 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3926 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049421+0000", 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049421+0000", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049422+0000", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0399999999999995e-07 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049426+0000", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3420000000000001e-06 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049437+0000", 2024-05-08T03:19:36.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0625000000000001e-05 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049450+0000", 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2993000000000001e-05 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049501+0000", 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0775000000000001e-05 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049579+0000", 2024-05-08T03:19:36.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8374000000000006e-05 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.049724+0000", 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144392 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.051895+0000", 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021713090000000002 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.054585+0000", 2024-05-08T03:19:36.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026899290000000002 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.054642+0000", 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7534000000000002e-05 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.054681+0000", 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8906000000000003e-05 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11575 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:8 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.693721+0000", 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.97388010300000005, 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0052220900000000004, 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11575 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693721+0000", 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693721+0000", 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693721+0000", 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7e-07 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693723+0000", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9860000000000001e-06 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693729+0000", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4419999999999997e-06 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693734+0000", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5979999999999999e-06 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.698515+0000", 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004781413 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.698591+0000", 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5872999999999997e-05 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.698943+0000", 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035230800000000002 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11576 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:8 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.693739+0000", 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.97386164900000005, 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0056718599999999999, 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11576 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693739+0000", 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693739+0000", 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693740+0000", 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693744+0000", 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3030000000000001e-06 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693750+0000", 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8610000000000003e-06 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693754+0000", 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3839999999999999e-06 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.698972+0000", 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0052175169999999996 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.699027+0000", 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5177999999999999e-05 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.699411+0000", 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00038420700000000002 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11577 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:8 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.693761+0000", 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.97384037099999998, 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0061346350000000003, 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11577 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693761+0000", 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693761+0000", 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693761+0000", 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693763+0000", 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1610000000000001e-06 2024-05-08T03:19:36.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693769+0000", 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693773+0000", 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1949999999999996e-06 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.699479+0000", 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057058650000000001 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.699542+0000", 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2544000000000003e-05 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.699895+0000", 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000353791 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11578 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:8 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.693779+0000", 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.97382172700000003, 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0067333970000000003, 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11578 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693779+0000", 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693779+0000", 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693780+0000", 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693782+0000", 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4389999999999999e-06 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693788+0000", 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8629999999999996e-06 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693797+0000", 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1130000000000007e-06 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.699937+0000", 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0061396740000000003 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.700073+0000", 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013616999999999999 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.700513+0000", 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000439741 2024-05-08T03:19:36.713 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11579 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:8 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.693803+0000", 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.97379825499999995, 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0071813060000000001, 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.714 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11579 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693803+0000", 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693803+0000", 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693803+0000", 2024-05-08T03:19:36.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2899999999999999e-07 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693805+0000", 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0949999999999998e-06 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693811+0000", 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.57e-06 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693815+0000", 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5310000000000003e-06 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.700547+0000", 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0067316779999999996 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.700635+0000", 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7529999999999997e-05 2024-05-08T03:19:36.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.700984+0000", 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000349573 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.720 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.721 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11580 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:8 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.721 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.693821+0000", 2024-05-08T03:19:36.721 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.97378048699999997, 2024-05-08T03:19:36.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075632750000000004, 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.722 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11580 2024-05-08T03:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693821+0000", 2024-05-08T03:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693821+0000", 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693821+0000", 2024-05-08T03:19:36.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3200000000000001e-07 2024-05-08T03:19:36.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693823+0000", 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-06 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693829+0000", 2024-05-08T03:19:36.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5269999999999998e-06 2024-05-08T03:19:36.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.693833+0000", 2024-05-08T03:19:36.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3100000000000002e-06 2024-05-08T03:19:36.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.701019+0000", 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0071863430000000004 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.701091+0000", 2024-05-08T03:19:36.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1441999999999998e-05 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.701384+0000", 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00029314100000000002 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.729 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11826 3.a 3:5299480c:::rbd_data.1182d7df86af.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.95394426099999996, 2024-05-08T03:19:36.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.009827967, 2024-05-08T03:19:36.731 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.731 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.731 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:36.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.732 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11826 2024-05-08T03:19:36.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.732 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713659+0000", 2024-05-08T03:19:36.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:19:36.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715527+0000", 2024-05-08T03:19:36.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001868562 2024-05-08T03:19:36.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715543+0000", 2024-05-08T03:19:36.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5341000000000001e-05 2024-05-08T03:19:36.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715550+0000", 2024-05-08T03:19:36.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4329999999999998e-06 2024-05-08T03:19:36.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715605+0000", 2024-05-08T03:19:36.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4818999999999998e-05 2024-05-08T03:19:36.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715701+0000", 2024-05-08T03:19:36.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6454999999999997e-05 2024-05-08T03:19:36.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715922+0000", 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00022026700000000001 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:36.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.719967+0000", 2024-05-08T03:19:36.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040455710000000004 2024-05-08T03:19:36.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:36.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723322+0000", 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003355051 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723454+0000", 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013202099999999999 2024-05-08T03:19:36.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723485+0000", 2024-05-08T03:19:36.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0580000000000002e-05 2024-05-08T03:19:36.742 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.742 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11864 3.a 3:57b2148a:::rbd_data.1182d7df86af.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.767384+0000", 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.90021669900000001, 2024-05-08T03:19:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.007501087, 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.744 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11864 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767384+0000", 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767384+0000", 2024-05-08T03:19:36.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767386+0000", 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.172e-06 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767391+0000", 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8340000000000004e-06 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767408+0000", 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6905000000000002e-05 2024-05-08T03:19:36.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767415+0000", 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767469+0000", 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4823e-05 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767554+0000", 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4336e-05 2024-05-08T03:19:36.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.767656+0000", 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00010188600000000001 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.768210+0000", 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00055487899999999998 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.774821+0000", 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0066100639999999997 2024-05-08T03:19:36.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.774853+0000", 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2882000000000001e-05 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.774886+0000", 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2119000000000002e-05 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.750 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12065 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:head [sparse-read 0~5636096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.65380312699999998, 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01338435, 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12065 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013800+0000", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.53e-06 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013807+0000", 2024-05-08T03:19:36.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5310000000000004e-06 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013825+0000", 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7459999999999999e-05 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013833+0000", 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013900+0000", 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7475e-05 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013998+0000", 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405000000000007e-05 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.027182+0000", 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013184695 2024-05-08T03:19:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12381 3.a 3:51f8ea84:::rbd_data.1182d7df86af.0000000000000018:9 [sparse-read 458752~163840] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.194364+0000", 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.47323702499999998, 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0055346010000000001, 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12381 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194364+0000", 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194364+0000", 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194364+0000", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9099999999999999e-07 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194367+0000", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.215e-06 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194373+0000", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5540000000000003e-06 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194383+0000", 2024-05-08T03:19:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7620000000000007e-06 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.199412+0000", 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050288319999999996 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.199461+0000", 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9292999999999997e-05 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.199899+0000", 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00043755399999999999 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12382 3.a 3:51f8ea84:::rbd_data.1182d7df86af.0000000000000018:9 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.194390+0000", 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.47321072199999997, 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057035899999999997, 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12382 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194390+0000", 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194390+0000", 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194391+0000", 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194393+0000", 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3029999999999998e-06 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194406+0000", 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194412+0000", 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.199939+0000", 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0055269860000000002 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200019+0000", 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9671000000000003e-05 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200094+0000", 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5426999999999995e-05 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12406 3.a 3:5299480c:::rbd_data.1182d7df86af.000000000000000b:9 [sparse-read 786432~163840] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.194949+0000", 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.47265252299999999, 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058632629999999996, 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12406 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194949+0000", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194949+0000", 2024-05-08T03:19:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194949+0000", 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8899999999999998e-07 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194954+0000", 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4719999999999997e-06 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194965+0000", 2024-05-08T03:19:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1226999999999999e-05 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194976+0000", 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1173e-05 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200135+0000", 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051588629999999996 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200207+0000", 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1583999999999998e-05 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200812+0000", 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00060505499999999996 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12407 3.a 3:5299480c:::rbd_data.1182d7df86af.000000000000000b:9 [sparse-read 1769472~163840] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.194996+0000", 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.47260544199999999, 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0066425149999999999, 2024-05-08T03:19:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12407 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194996+0000", 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194996+0000", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194996+0000", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7999999999999995e-07 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.194999+0000", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7259999999999998e-06 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195007+0000", 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4619999999999999e-06 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195012+0000", 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200855+0000", 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058430729999999998 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.200913+0000", 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7732000000000002e-05 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.201638+0000", 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00072566700000000004 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12432 3.a 3:51f8ea84:::rbd_data.1182d7df86af.0000000000000018:9 [sparse-read 622592~163840] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.195071+0000", 2024-05-08T03:19:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.47253041600000001, 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0071989059999999997, 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12432 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195071+0000", 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195071+0000", 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195071+0000", 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4700000000000001e-07 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195075+0000", 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.292e-06 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195084+0000", 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2410000000000001e-06 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195090+0000", 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4060000000000002e-06 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.201681+0000", 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.006590434 2024-05-08T03:19:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.201758+0000", 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7791000000000001e-05 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.202270+0000", 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00051119499999999999 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12433 3.a 3:51f8ea84:::rbd_data.1182d7df86af.0000000000000018:9 [sparse-read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.195100+0000", 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.472500955, 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0073453989999999999, 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12433 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195100+0000", 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195100+0000", 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195101+0000", 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195103+0000", 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5490000000000001e-06 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195110+0000", 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0879999999999997e-06 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.195116+0000", 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.271e-06 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.202318+0000", 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0072026879999999996 2024-05-08T03:19:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.202373+0000", 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000002e-05 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.202446+0000", 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2713999999999995e-05 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.45918176500000002, 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:36.785 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:36.787 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:36.787 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:36.785+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:36.862 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:19:36.887 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:36.885+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:36.920 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone7 from 0x9e0caa8 (overlap 0x43343a9) to 0xd70d88 2024-05-08T03:19:36.944 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:36.944 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:36.944 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:36.944 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:36.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:36.982 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-05-08T03:19:36.998 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:36.994+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:37.073 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.073 INFO:teuthology.orchestra.run.smithi177.stdout:610 read 0x1871936 thru 0x187a0c3 (0x878e bytes) 2024-05-08T03:19:37.074 INFO:teuthology.orchestra.run.smithi177.stdout:612 read 0x3363e78 thru 0x33688e6 (0x4a6f bytes) 2024-05-08T03:19:37.075 INFO:teuthology.orchestra.run.smithi177.stdout:613 write 0x94c2a93 thru 0x94ca5e3 (0x7b51 bytes) 2024-05-08T03:19:37.083 INFO:teuthology.orchestra.run.smithi177.stdout:614 read 0x51f25b thru 0x52972c (0xa4d2 bytes) 2024-05-08T03:19:37.083 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.083 INFO:teuthology.orchestra.run.smithi177.stdout:616 punch from 0x11f6248 to 0x11fe78c, (0x8544 bytes) 2024-05-08T03:19:37.084 INFO:teuthology.orchestra.run.smithi177.stdout:617 punch from 0x28132ad to 0x28137dc, (0x52f bytes) 2024-05-08T03:19:37.084 INFO:teuthology.orchestra.run.smithi177.stdout:620 writesame 0x34de11d thru 0x34e0d95 data_size 0x2d(0x2c79 bytes) 2024-05-08T03:19:37.089 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:37.087+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:37.100 INFO:teuthology.orchestra.run.smithi177.stdout:622 write 0xde18f5e thru 0xde24647 (0xb6ea bytes) 2024-05-08T03:19:37.105 INFO:teuthology.orchestra.run.smithi177.stdout:623 punch from 0x710a91e to 0x7113e81, (0x9563 bytes) 2024-05-08T03:19:37.107 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:37.107 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:37.107 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:37.107 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:37.107 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:37.116 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:37.121 INFO:teuthology.orchestra.run.smithi177.stdout:624 trunc from 0xde24648 to 0xcf69a1 2024-05-08T03:19:37.190 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:37.188+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:37.205 INFO:teuthology.orchestra.run.smithi177.stdout:625 writesame 0xe9bf4c3 thru 0xe9cef8b data_size 0x1f(0xfac9 bytes) 2024-05-08T03:19:37.205 INFO:teuthology.orchestra.run.smithi177.stdout:626 writesame 0x27421f thru 0x27fa61 data_size 0x1(0xb843 bytes) 2024-05-08T03:19:37.227 INFO:teuthology.orchestra.run.smithi177.stdout:627 writesame 0x8c789da thru 0x8c7ec59 data_size 0x40(0x6280 bytes) 2024-05-08T03:19:37.231 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.231 INFO:teuthology.orchestra.run.smithi177.stdout:632 read 0x1e9183e thru 0x1e9e5d7 (0xcd9a bytes) 2024-05-08T03:19:37.231 INFO:teuthology.orchestra.run.smithi177.stdout:633 writesame 0xcefe8ca thru 0xcf04bf6 data_size 0x27(0x632d bytes) 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.357921363, 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:19:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:19:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:19:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:19:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0159801709999998, 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0159647720000002, 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:19:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0159331299999996, 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0159197979999997, 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:19:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:19:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0159100040000002, 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0159005910000003, 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:19:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:19:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158910700000003, 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158817290000002, 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158725139999998, 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:19:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158631519999997, 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:19:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.01585372, 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:19:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:19:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158443039999998, 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:19:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:19:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158352859999997, 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158262430000002, 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:19:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158139500000001, 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:19:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:19:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0158046829999998, 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0157908249999998, 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.307 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.0157815320000001, 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.311 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:19:37.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.315 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 1.0227673989999999, 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:37.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:37.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:37.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.319 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:37.320 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:37.320 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:37.322 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:37.322 INFO:teuthology.orchestra.run.smithi177.stdout:634 write 0x8b2c2e5 thru 0x8b38a82 (0xc79e bytes) 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:635 read 0xd16bfa5 thru 0xd174530 (0x858c bytes) 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:636 write 0xd995065 thru 0xd9992b4 (0x4250 bytes) 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:637 write 0xc240d3b thru 0xc24f930 (0xebf6 bytes) 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:638 punch from 0xc587531 to 0xc59520f, (0xdcde bytes) 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:639 trunc from 0xe9cef8c to 0xdea7b87 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:640 write 0x9f9f345 thru 0x9fa739d (0x8059 bytes) 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:641 trunc from 0xdea7b87 to 0x9af59c0 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:642 trunc from 0x9af59c0 to 0x3a93bf1 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.323 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.323 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:37.319+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:37.324 INFO:teuthology.orchestra.run.smithi177.stdout:648 write 0x4a9ce9b thru 0x4aaa89b (0xda01 bytes) 2024-05-08T03:19:37.329 INFO:teuthology.orchestra.run.smithi177.stdout:651 read 0x3358cc3 thru 0x335e427 (0x5765 bytes) 2024-05-08T03:19:37.345 INFO:teuthology.orchestra.run.smithi177.stdout:652 writesame 0xd151a55 thru 0xd158ead data_size 0x1(0x7459 bytes) 2024-05-08T03:19:37.345 INFO:teuthology.orchestra.run.smithi177.stdout:653 write 0x9929df1 thru 0x9930b33 (0x6d43 bytes) 2024-05-08T03:19:37.348 INFO:teuthology.orchestra.run.smithi177.stdout:655 write 0x2e8c6c7 thru 0x2e988f7 (0xc231 bytes) 2024-05-08T03:19:37.352 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.352 INFO:teuthology.orchestra.run.smithi177.stdout:659 write 0xcb9f600 thru 0xcbac575 (0xcf76 bytes) 2024-05-08T03:19:37.357 INFO:teuthology.orchestra.run.smithi177.stdout:660 punch from 0x2067aa9 to 0x2075ca7, (0xe1fe bytes) 2024-05-08T03:19:37.364 INFO:teuthology.orchestra.run.smithi177.stdout:661 writesame 0xec57ceb thru 0xec5bfcb data_size 0xd(0x42e1 bytes) 2024-05-08T03:19:37.364 INFO:teuthology.orchestra.run.smithi177.stdout:664 read 0x717da79 thru 0x7181b65 (0x40ed bytes) 2024-05-08T03:19:37.364 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.364 INFO:teuthology.orchestra.run.smithi177.stdout:668 punch from 0xab12ea6 to 0xab20bc2, (0xdd1c bytes) 2024-05-08T03:19:37.364 INFO:teuthology.orchestra.run.smithi177.stdout:669 punch from 0x58067fe to 0x580ebed, (0x83ef bytes) 2024-05-08T03:19:37.379 INFO:teuthology.orchestra.run.smithi177.stdout:670 trunc from 0xec5bfcc to 0x206dd41 2024-05-08T03:19:37.396 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.396 INFO:teuthology.orchestra.run.smithi177.stdout:673 read 0x12cd27d thru 0x12ceb57 (0x18db bytes) 2024-05-08T03:19:37.396 INFO:teuthology.orchestra.run.smithi177.stdout:674 write 0xc99508 thru 0xca352e (0xa027 bytes) 2024-05-08T03:19:37.398 INFO:teuthology.orchestra.run.smithi177.stdout:675 write 0xbc5214 thru 0xbca461 (0x524e bytes) 2024-05-08T03:19:37.399 INFO:teuthology.orchestra.run.smithi177.stdout:676 read 0x1036d6a thru 0x103a2f1 (0x3588 bytes) 2024-05-08T03:19:37.399 INFO:teuthology.orchestra.run.smithi177.stdout:677 read 0x1854385 thru 0x185e4de (0xa15a bytes) 2024-05-08T03:19:37.409 INFO:teuthology.orchestra.run.smithi177.stdout:678 write 0x537ce13 thru 0x5383510 (0x66fe bytes) 2024-05-08T03:19:37.418 INFO:teuthology.orchestra.run.smithi177.stdout:680 write 0x32715c7 thru 0x3271b9b (0x5d5 bytes) 2024-05-08T03:19:37.421 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:37.419+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:37.422 INFO:teuthology.orchestra.run.smithi177.stdout:681 read 0x3cac096 thru 0x3cb4b75 (0x8ae0 bytes) 2024-05-08T03:19:37.423 INFO:teuthology.orchestra.run.smithi177.stdout:682 read 0xaf61db thru 0xb05c70 (0xfa96 bytes) 2024-05-08T03:19:37.423 INFO:teuthology.orchestra.run.smithi177.stdout:683 read 0x4f9e509 thru 0x4fa46b3 (0x61ab bytes) 2024-05-08T03:19:37.432 INFO:teuthology.orchestra.run.smithi177.stdout:684 writesame 0x8c05dfd thru 0x8c0d547 data_size 0x1(0x774b bytes) 2024-05-08T03:19:37.433 INFO:teuthology.orchestra.run.smithi177.stdout:685 read 0x498451e thru 0x498c3c9 (0x7eac bytes) 2024-05-08T03:19:37.433 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.433 INFO:teuthology.orchestra.run.smithi177.stdout:687 punch from 0x601001c to 0x6015348, (0x532c bytes) 2024-05-08T03:19:37.438 INFO:teuthology.orchestra.run.smithi177.stdout:688 trunc from 0x8c0d548 to 0x3ffa8f0 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13152 3.17 3:eda2629d:::rbd_data.118231ecabb9.000000000000002b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.442351+0000", 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0010811110000000001, 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001102121, 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.444 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for sub ops", 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13152 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442351+0000", 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442351+0000", 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442351+0000", 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8599999999999998e-07 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442354+0000", 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0730000000000001e-06 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442362+0000", 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4590000000000001e-06 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442366+0000", 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.301e-06 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442416+0000", 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9645999999999999e-05 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442494+0000", 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8603000000000006e-05 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442599+0000", 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000104663 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443256+0000", 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000657487 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.448 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13153 3.17 3:e99d1f62:::rbd_data.118231ecabb9.000000000000002c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.442386+0000", 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0010457109999999999, 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011267759999999999, 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for sub ops", 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13153 2024-05-08T03:19:37.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442386+0000", 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442386+0000", 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442386+0000", 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3500000000000002e-07 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442389+0000", 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3e-06 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442394+0000", 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.463e-06 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442399+0000", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0640000000000001e-06 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442875+0000", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000475441 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.442948+0000", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2912999999999998e-05 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443029+0000", 2024-05-08T03:19:37.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1454999999999999e-05 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443471+0000", 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00044235900000000002 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop_reply(client.4482.0:13152 3.17 e34/19 ondisk, result = 0)", 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.443509+0000", 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4294967295.9999223, 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5533999999999998e-05, 2024-05-08T03:19:37.453 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443509+0000", 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443509+0000", 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443510+0000", 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1200000000000002e-07 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443513+0000", 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999998e-06 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443523+0000", 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1700000000000003e-06 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443530+0000", 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0169999999999999e-06 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443557+0000", 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6991999999999998e-05 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.443561+0000", 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5909999999999998e-06 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.457 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.457 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:19:37.457 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 3 2024-05-08T03:19:37.457 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:37.458 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:37.458 INFO:teuthology.orchestra.run.smithi177.stdout:690 writesame 0x11c7068 thru 0x11cee1c data_size 0x11(0x7db5 bytes) 2024-05-08T03:19:37.459 INFO:teuthology.orchestra.run.smithi177.stdout:692 read 0xf6d773 thru 0xf76e76 (0x9704 bytes) 2024-05-08T03:19:37.461 INFO:teuthology.orchestra.run.smithi177.stdout:693 trunc from 0x3ffa8f0 to 0x1a2fa6e 2024-05-08T03:19:37.485 INFO:teuthology.orchestra.run.smithi177.stdout:694 trunc from 0x1a2fa6e to 0xb8db14b 2024-05-08T03:19:37.492 INFO:teuthology.orchestra.run.smithi177.stdout:695 trunc from 0xb8db14b to 0xcf51431 2024-05-08T03:19:37.498 INFO:teuthology.orchestra.run.smithi177.stdout:696 write 0x9a3dfee thru 0x9a4474f (0x6762 bytes) 2024-05-08T03:19:37.503 INFO:teuthology.orchestra.run.smithi177.stdout:697 punch from 0xc3871f5 to 0xc3907bf, (0x95ca bytes) 2024-05-08T03:19:37.503 INFO:teuthology.orchestra.run.smithi177.stdout:698 writesame 0x5eda594 thru 0x5ee44cb data_size 0x14(0x9f38 bytes) 2024-05-08T03:19:37.512 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.512 INFO:teuthology.orchestra.run.smithi177.stdout:700 trunc from 0xcf51431 to 0xcbcfb05 2024-05-08T03:19:37.518 INFO:teuthology.orchestra.run.smithi177.stdout:701 read 0x22fcc53 thru 0x2303080 (0x642e bytes) 2024-05-08T03:19:37.518 INFO:teuthology.orchestra.run.smithi177.stdout:703 write 0x5a7e750 thru 0x5a822e6 (0x3b97 bytes) 2024-05-08T03:19:37.523 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:37.521+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:37.524 INFO:teuthology.orchestra.run.smithi177.stdout:704 read 0x3319d31 thru 0x33261ef (0xc4bf bytes) 2024-05-08T03:19:37.524 INFO:teuthology.orchestra.run.smithi177.stdout:705 read 0x330b2c9 thru 0x330dce6 (0x2a1e bytes) 2024-05-08T03:19:37.524 INFO:teuthology.orchestra.run.smithi177.stdout:707 trunc from 0xcbcfb05 to 0xcb55798 2024-05-08T03:19:37.535 INFO:teuthology.orchestra.run.smithi177.stdout:709 trunc from 0xcb55798 to 0x8930254 2024-05-08T03:19:37.544 INFO:teuthology.orchestra.run.smithi177.stdout:710 punch from 0x23d5146 to 0x23e50e1, (0xff9b bytes) 2024-05-08T03:19:37.544 INFO:teuthology.orchestra.run.smithi177.stdout:711 write 0x6798131 thru 0x67a37a7 (0xb677 bytes) 2024-05-08T03:19:37.551 INFO:teuthology.orchestra.run.smithi177.stdout:712 punch from 0x55dd710 to 0x55e9485, (0xbd75 bytes) 2024-05-08T03:19:37.551 INFO:teuthology.orchestra.run.smithi177.stdout:714 read 0x3373d6d thru 0x337702b (0x32bf bytes) 2024-05-08T03:19:37.560 INFO:teuthology.orchestra.run.smithi177.stdout:717 writesame 0xc271ac6 thru 0xc277b6f data_size 0x1(0x60aa bytes) 2024-05-08T03:19:37.560 INFO:teuthology.orchestra.run.smithi177.stdout:718 writesame 0x6454adc thru 0x6463935 data_size 0x1(0xee5a bytes) 2024-05-08T03:19:37.582 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:37.582 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:37.582 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:37.582 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:37.582 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:37.600 INFO:teuthology.orchestra.run.smithi177.stdout:719 write 0xa5ab1c1 thru 0xa5ba697 (0xf4d7 bytes) 2024-05-08T03:19:37.601 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:37.602 INFO:teuthology.orchestra.run.smithi177.stdout:723 read 0xa5ebe5 thru 0xa62a38 (0x3e54 bytes) 2024-05-08T03:19:37.602 INFO:teuthology.orchestra.run.smithi177.stdout:724 writesame 0xb2f00ab thru 0xb2f6e65 data_size 0x1(0x6dbb bytes) 2024-05-08T03:19:37.604 INFO:teuthology.orchestra.run.smithi177.stdout:726 read 0x337d88d thru 0x338a2ae (0xca22 bytes) 2024-05-08T03:19:37.605 INFO:teuthology.orchestra.run.smithi177.stdout:728 writesame 0xc25dfea thru 0xc26bea2 data_size 0x25(0xdeb9 bytes) 2024-05-08T03:19:37.614 INFO:teuthology.orchestra.run.smithi177.stdout:729 read 0x7160461 thru 0x716bcaf (0xb84f bytes) 2024-05-08T03:19:37.614 INFO:teuthology.orchestra.run.smithi177.stdout:731 read 0xb09bfe9 thru 0xb0a4d1a (0x8d32 bytes) 2024-05-08T03:19:37.614 INFO:teuthology.orchestra.run.smithi177.stdout:733 write 0xa7176c thru 0xa7522a (0x3abf bytes) 2024-05-08T03:19:37.614 INFO:teuthology.orchestra.run.smithi177.stdout:734 read 0xb7038c9 thru 0xb7102cc (0xca04 bytes) 2024-05-08T03:19:37.614 INFO:teuthology.orchestra.run.smithi177.stdout:735 read 0xea2e29 thru 0xeaf3c2 (0xc59a bytes) 2024-05-08T03:19:37.615 INFO:teuthology.orchestra.run.smithi177.stdout:737 writesame 0xb456b40 thru 0xb4581ee data_size 0x1(0x16af bytes) 2024-05-08T03:19:37.623 INFO:teuthology.orchestra.run.smithi177.stdout:738 write 0x1e75810 thru 0x1e7e718 (0x8f09 bytes) 2024-05-08T03:19:37.623 INFO:teuthology.orchestra.run.smithi177.stdout:739 read 0x2fb42fb thru 0x2fb8b8e (0x4894 bytes) 2024-05-08T03:19:37.625 INFO:teuthology.orchestra.run.smithi177.stdout:740 read 0x190d1c1 thru 0x190fb10 (0x2950 bytes) 2024-05-08T03:19:37.625 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.625 INFO:teuthology.orchestra.run.smithi177.stdout:742 write 0x3976382 thru 0x3984311 (0xdf90 bytes) 2024-05-08T03:19:37.626 INFO:teuthology.orchestra.run.smithi177.stdout:744 write 0x4f4fc0d thru 0x4f5eab6 (0xeeaa bytes) 2024-05-08T03:19:37.626 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:37.621+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:37.629 INFO:teuthology.orchestra.run.smithi177.stdout:745 write 0x5ff6cb6 thru 0x5ffcb75 (0x5ec0 bytes) 2024-05-08T03:19:37.631 INFO:teuthology.orchestra.run.smithi177.stdout:746 writesame 0xbe0c8c thru 0xbea1fd data_size 0x2(0x9572 bytes) 2024-05-08T03:19:37.640 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.655 INFO:teuthology.orchestra.run.smithi177.stdout:749 trunc from 0xc277b70 to 0xbaba57 2024-05-08T03:19:37.677 INFO:teuthology.orchestra.run.smithi177.stdout:750 trunc from 0xbaba57 to 0x5fc3927 2024-05-08T03:19:37.683 INFO:teuthology.orchestra.run.smithi177.stdout:751 read 0x546aa4d thru 0x5475908 (0xaebc bytes) 2024-05-08T03:19:37.683 INFO:teuthology.orchestra.run.smithi177.stdout:753 punch from 0x1feff26 to 0x1ffe541, (0xe61b bytes) 2024-05-08T03:19:37.683 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.683 INFO:teuthology.orchestra.run.smithi177.stdout:755 write 0x1c542de thru 0x1c62098 (0xddbb bytes) 2024-05-08T03:19:37.686 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:37.687 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.692 INFO:teuthology.orchestra.run.smithi177.stdout:758 trunc from 0x5fc3927 to 0xf463f7 2024-05-08T03:19:37.722 INFO:teuthology.orchestra.run.smithi177.stdout:760 trunc from 0xf463f7 to 0xb917b3b 2024-05-08T03:19:37.724 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:37.722+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:37.727 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.727 INFO:teuthology.orchestra.run.smithi177.stdout:762 writesame 0xab076c8 thru 0xab149e2 data_size 0x11(0xd31b bytes) 2024-05-08T03:19:37.731 INFO:teuthology.orchestra.run.smithi177.stdout:764 read 0x67e35f1 thru 0x67f235f (0xed6f bytes) 2024-05-08T03:19:37.731 INFO:teuthology.orchestra.run.smithi177.stdout:765 write 0xa0b9375 thru 0xa0c8d26 (0xf9b2 bytes) 2024-05-08T03:19:37.736 INFO:teuthology.orchestra.run.smithi177.stdout:766 read 0x9fd0aa9 thru 0x9fde986 (0xdede bytes) 2024-05-08T03:19:37.736 INFO:teuthology.orchestra.run.smithi177.stdout:767 read 0x3088a86 thru 0x308b981 (0x2efc bytes) 2024-05-08T03:19:37.736 INFO:teuthology.orchestra.run.smithi177.stdout:768 trunc from 0xb917b3b to 0xca7cd87 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5387668169999991, 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.741 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:37.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:19:37.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5387573670000005, 2024-05-08T03:19:37.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:37.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:19:37.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5387446310000001, 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.753 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:19:37.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:19:37.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:37.758 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5387340550000008, 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:37.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:19:37.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:37.763 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5387246680000004, 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.768 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5387154879999994, 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:19:37.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:19:37.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3078904530000006, 2024-05-08T03:19:37.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:19:37.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.778 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3078582159999996, 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:19:37.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.782 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3078448859999998, 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:19:37.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:37.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:19:37.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:37.787 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3078299120000008, 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:37.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3078186029999994, 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:19:37.792 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:19:37.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:19:37.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:19:37.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.797 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3078009339999994, 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3077857359999996, 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.802 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:19:37.806 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3077716420000005, 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:19:37.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.814 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5328176689999999, 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:19:37.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:19:37.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.60992386700000001, 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.821 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:37.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:19:37.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:19:37.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.609910379, 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.826 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.60988139799999996, 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.831 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:19:37.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.60985479200000003, 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:19:37.836 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:37.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:19:37.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:37.841 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.17710653100000001, 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:37.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:19:37.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:19:37.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:37.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:19:37.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:37.848 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:37.848 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:37.849 INFO:teuthology.orchestra.run.smithi177.stdout:769 read 0x643fe37 thru 0x6449fea (0xa1b4 bytes) 2024-05-08T03:19:37.849 INFO:teuthology.orchestra.run.smithi177.stdout:771 read 0x2a7b57b thru 0x2a82b75 (0x75fb bytes) 2024-05-08T03:19:37.849 INFO:teuthology.orchestra.run.smithi177.stdout:772 trunc from 0xca7cd87 to 0x4752561 2024-05-08T03:19:37.850 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:775 write 0x1d8ad7f thru 0x1d91da6 (0x7028 bytes) 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:776 write 0xcf4e30c thru 0xcf54495 (0x618a bytes) 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:777 punch from 0xc2591ef to 0xc25f4a9, (0x62ba bytes) 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:780 read 0x6c87915 thru 0x6c91b3c (0xa228 bytes) 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:781 trunc from 0xcf54496 to 0x60bceae 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:782 trunc from 0x60bceae to 0x518a3a7 2024-05-08T03:19:37.852 INFO:teuthology.orchestra.run.smithi177.stdout:784 read 0x13bdd63 thru 0x13cc954 (0xebf2 bytes) 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stdout:785 punch from 0x4750bd3 to 0x47520f3, (0x1520 bytes) 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stdout:787 punch from 0x4e15b2e to 0x4e170cd, (0x159f bytes) 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stdout:789 write 0xd441346 thru 0xd44c28f (0xaf4a bytes) 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stdout:790 read 0x6353f28 thru 0x635b336 (0x740f bytes) 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stdout:791 writesame 0x223626b thru 0x223b662 data_size 0x4(0x53f8 bytes) 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:37.853 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:37.854 INFO:teuthology.orchestra.run.smithi177.stdout:792 writesame 0xd01abd5 thru 0xd0280f1 data_size 0x1(0xd51d bytes) 2024-05-08T03:19:37.854 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:37.847+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:37.855 INFO:teuthology.orchestra.run.smithi177.stderr:set 3.d pg_upmap mapping to [3,6] 2024-05-08T03:19:37.859 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:19:37.861 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:37.862 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:37.864 INFO:teuthology.orchestra.run.smithi177.stdout:793 write 0xc171877 thru 0xc17cbdf (0xb369 bytes) 2024-05-08T03:19:37.868 INFO:teuthology.orchestra.run.smithi177.stdout:795 write 0x2ba0c94 thru 0x2ba4f67 (0x42d4 bytes) 2024-05-08T03:19:37.873 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.886 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:37.886 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:37.886 INFO:teuthology.orchestra.run.smithi177.stdout:797 trunc from 0xd44c290 to 0x3c0212a 2024-05-08T03:19:37.897 INFO:teuthology.orchestra.run.smithi177.stdout:799 read 0x19480d6 thru 0x194a330 (0x225b bytes) 2024-05-08T03:19:37.911 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:37.912 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:37.913 INFO:teuthology.orchestra.run.smithi177.stdout:800 writesame 0xb377c59 thru 0xb3810eb data_size 0x1(0x9493 bytes) 2024-05-08T03:19:37.920 INFO:teuthology.orchestra.run.smithi177.stdout:803 trunc from 0xb3810ec to 0x6c44309 2024-05-08T03:19:37.926 INFO:teuthology.orchestra.run.smithi177.stdout:804 read 0x4eafa1b thru 0x4eb8d48 (0x932e bytes) 2024-05-08T03:19:37.926 INFO:teuthology.orchestra.run.smithi177.stdout:805 write 0x57de1d8 thru 0x57df839 (0x1662 bytes) 2024-05-08T03:19:37.929 INFO:teuthology.orchestra.run.smithi177.stdout:806 writesame 0x2aac189 thru 0x2aae275 data_size 0x1(0x20ed bytes) 2024-05-08T03:19:37.934 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:37.934 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:37.935 INFO:teuthology.orchestra.run.smithi177.stdout:807 read 0x2ee9ded thru 0x2ef31d2 (0x93e6 bytes) 2024-05-08T03:19:37.935 INFO:teuthology.orchestra.run.smithi177.stdout:808 read 0x3847caf thru 0x384bb55 (0x3ea7 bytes) 2024-05-08T03:19:37.949 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:37.948+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:37.950 INFO:teuthology.orchestra.run.smithi177.stdout:809 write 0xeb2385c thru 0xeb3289f (0xf044 bytes) 2024-05-08T03:19:37.952 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:37.953 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:37.954 INFO:teuthology.orchestra.run.smithi177.stdout:810 write 0x209ecdf thru 0x20a2a29 (0x3d4b bytes) 2024-05-08T03:19:37.955 INFO:teuthology.orchestra.run.smithi177.stdout:811 punch from 0xd0ddd7b to 0xd0ed5e7, (0xf86c bytes) 2024-05-08T03:19:37.955 INFO:teuthology.orchestra.run.smithi177.stdout:812 write 0x316067 thru 0x323956 (0xd8f0 bytes) 2024-05-08T03:19:37.957 INFO:teuthology.orchestra.run.smithi177.stdout:813 write 0x18ca0bf thru 0x18ce8b1 (0x47f3 bytes) 2024-05-08T03:19:37.960 INFO:teuthology.orchestra.run.smithi177.stdout:816 punch from 0xb4eae7f to 0xb4f9f6c, (0xf0ed bytes) 2024-05-08T03:19:37.960 INFO:teuthology.orchestra.run.smithi177.stdout:817 writesame 0x9dc19fb thru 0x9dc4c42 data_size 0x4(0x3248 bytes) 2024-05-08T03:19:37.973 INFO:teuthology.orchestra.run.smithi177.stdout:819 trunc from 0xeb328a0 to 0x74005b9 2024-05-08T03:19:37.985 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:37.985 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:37.985 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:37.986 INFO:teuthology.orchestra.run.smithi177.stdout:824 read 0x91e216 thru 0x92ac7d (0xca68 bytes) 2024-05-08T03:19:37.993 INFO:teuthology.orchestra.run.smithi177.stdout:828 write 0x849e485 thru 0x84a5a5e (0x75da bytes) 2024-05-08T03:19:37.996 INFO:teuthology.orchestra.run.smithi177.stdout:829 write 0x28490ca thru 0x28542c4 (0xb1fb bytes) 2024-05-08T03:19:37.997 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:37.998 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:37.999 INFO:teuthology.orchestra.run.smithi177.stdout:830 read 0x2bbf91f thru 0x2bc7c6d (0x834f bytes) 2024-05-08T03:19:38.000 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:38.006 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:38.006 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:38.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:38.009 INFO:teuthology.orchestra.run.smithi177.stdout:833 write 0x8df4c45 thru 0x8df7f01 (0x32bd bytes) 2024-05-08T03:19:38.012 INFO:teuthology.orchestra.run.smithi177.stdout:835 punch from 0x632e4ef to 0x633ba1e, (0xd52f bytes) 2024-05-08T03:19:38.012 INFO:teuthology.orchestra.run.smithi177.stdout:836 write 0x4de5410 thru 0x4de88b4 (0x34a5 bytes) 2024-05-08T03:19:38.018 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:38.019 INFO:teuthology.orchestra.run.smithi177.stdout:839 write 0x48c1a01 thru 0x48cd2d4 (0xb8d4 bytes) 2024-05-08T03:19:38.019 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:38.019 INFO:teuthology.orchestra.run.smithi177.stdout:841 read 0x48aea3 thru 0x49858f (0xd6ed bytes) 2024-05-08T03:19:38.019 INFO:teuthology.orchestra.run.smithi177.stdout:842 punch from 0x6a96abf to 0x6a9bcbc, (0x51fd bytes) 2024-05-08T03:19:38.019 INFO:teuthology.orchestra.run.smithi177.stdout:843 read 0x61db221 thru 0x61e2bee (0x79ce bytes) 2024-05-08T03:19:38.020 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:38.020 INFO:teuthology.orchestra.run.smithi177.stdout:845 write 0x4c8df09 thru 0x4c90e2a (0x2f22 bytes) 2024-05-08T03:19:38.021 INFO:teuthology.orchestra.run.smithi177.stdout:846 read 0x12ebe82 thru 0x12f2645 (0x67c4 bytes) 2024-05-08T03:19:38.022 INFO:teuthology.orchestra.run.smithi177.stdout:847 read 0x30fd43e thru 0x3107f15 (0xaad8 bytes) 2024-05-08T03:19:38.022 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:38.022 INFO:teuthology.orchestra.run.smithi177.stdout:855 write 0x4bff77e thru 0x4c02598 (0x2e1b bytes) 2024-05-08T03:19:38.027 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:38.027 INFO:teuthology.orchestra.run.smithi177.stdout:858 read 0x2baa210 thru 0x2bacc69 (0x2a5a bytes) 2024-05-08T03:19:38.028 INFO:teuthology.orchestra.run.smithi177.stdout:859 write 0x351b637 thru 0x351ec2a (0x35f4 bytes) 2024-05-08T03:19:38.050 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:38.049+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:38.117 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.117 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.126 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:38.168 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:38.166+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:38.252 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:38.251+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:38.273 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.274 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.279 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:38.354 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:38.352+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:38.384 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.384 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.393 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:38.455 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:38.454+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:38.498 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.499 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.507 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:38.556 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:38.555+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:38.613 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.613 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.622 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:38.656 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:38.655+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:38.757 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:38.756+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:38.768 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.768 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.778 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:38.858 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:38.857+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:38.925 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:38.925 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:38.934 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:38.978 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:38.977+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:39.040 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.040 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.050 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:39.094 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:39.092+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:39.101 INFO:teuthology.orchestra.run.smithi177.stdout:861 trunc from 0x8df7f02 to 0x781ddc5 2024-05-08T03:19:39.129 INFO:teuthology.orchestra.run.smithi177.stdout:862 trunc from 0x781ddc5 to 0x5b11fc8 2024-05-08T03:19:39.139 INFO:teuthology.orchestra.run.smithi177.stdout:863 read 0x12c3ac3 thru 0x12cbeb0 (0x83ee bytes) 2024-05-08T03:19:39.139 INFO:teuthology.orchestra.run.smithi177.stdout:864 write 0x10e0f62 thru 0x10ef841 (0xe8e0 bytes) 2024-05-08T03:19:39.141 INFO:teuthology.orchestra.run.smithi177.stdout:865 read 0x3210684 thru 0x321abce (0xa54b bytes) 2024-05-08T03:19:39.141 INFO:teuthology.orchestra.run.smithi177.stdout:866 read 0x2080ece thru 0x208ca99 (0xbbcc bytes) 2024-05-08T03:19:39.142 INFO:teuthology.orchestra.run.smithi177.stdout:867 punch from 0x1aa2c9e to 0x1aaee80, (0xc1e2 bytes) 2024-05-08T03:19:39.148 INFO:teuthology.orchestra.run.smithi177.stdout:869 write 0x646f0d2 thru 0x6470187 (0x10b6 bytes) 2024-05-08T03:19:39.161 INFO:teuthology.orchestra.run.smithi177.stdout:870 trunc from 0x6470188 to 0x209af4 2024-05-08T03:19:39.161 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:39.159+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:39.177 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.177 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:39.199 INFO:teuthology.orchestra.run.smithi177.stdout:871 write 0x7452b74 thru 0x7455447 (0x28d4 bytes) 2024-05-08T03:19:39.202 INFO:teuthology.orchestra.run.smithi177.stdout:872 read 0x52367f1 thru 0x523cb25 (0x6335 bytes) 2024-05-08T03:19:39.202 INFO:teuthology.orchestra.run.smithi177.stdout:874 punch from 0x739d74b to 0x73ab2cd, (0xdb82 bytes) 2024-05-08T03:19:39.209 INFO:teuthology.orchestra.run.smithi177.stdout:875 write 0x81fef8e thru 0x8207a76 (0x8ae9 bytes) 2024-05-08T03:19:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:876 punch from 0x76d85d0 to 0x76dfa35, (0x7465 bytes) 2024-05-08T03:19:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:878 punch from 0x81510fc to 0x81556be, (0x45c2 bytes) 2024-05-08T03:19:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:879 read 0x81cb7be thru 0x81d8c20 (0xd463 bytes) 2024-05-08T03:19:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:880 read 0x3a8702b thru 0x3a93ec8 (0xce9e bytes) 2024-05-08T03:19:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:882 write 0x2708b0c thru 0x270a4a0 (0x1995 bytes) 2024-05-08T03:19:39.219 INFO:teuthology.orchestra.run.smithi177.stdout:884 read 0x44f14b3 thru 0x44f6d21 (0x586f bytes) 2024-05-08T03:19:39.220 INFO:teuthology.orchestra.run.smithi177.stdout:885 read 0x2d851e0 thru 0x2d8e2e1 (0x9102 bytes) 2024-05-08T03:19:39.231 INFO:teuthology.orchestra.run.smithi177.stdout:886 write 0xa967c29 thru 0xa97470a (0xcae2 bytes) 2024-05-08T03:19:39.235 INFO:teuthology.orchestra.run.smithi177.stdout:888 read 0x4fa00b6 thru 0x4fafc2f (0xfb7a bytes) 2024-05-08T03:19:39.235 INFO:teuthology.orchestra.run.smithi177.stdout:890 write 0xa3bf18 thru 0xa4b33b (0xf424 bytes) 2024-05-08T03:19:39.237 INFO:teuthology.orchestra.run.smithi177.stdout:891 punch from 0x229ce5a to 0x22a5e91, (0x9037 bytes) 2024-05-08T03:19:39.242 INFO:teuthology.orchestra.run.smithi177.stdout:892 writesame 0xaef7d49 thru 0xaefb8c7 data_size 0x1(0x3b7f bytes) 2024-05-08T03:19:39.243 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:39.243 INFO:teuthology.orchestra.run.smithi177.stdout:894 read 0x150e530 thru 0x15180d2 (0x9ba3 bytes) 2024-05-08T03:19:39.243 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:39.243 INFO:teuthology.orchestra.run.smithi177.stdout:896 write 0x4583cba thru 0x45906a4 (0xc9eb bytes) 2024-05-08T03:19:39.246 INFO:teuthology.orchestra.run.smithi177.stdout:897 write 0x5782dae thru 0x578bb41 (0x8d94 bytes) 2024-05-08T03:19:39.250 INFO:teuthology.orchestra.run.smithi177.stdout:898 write 0x6a01068 thru 0x6a05aff (0x4a98 bytes) 2024-05-08T03:19:39.252 INFO:teuthology.orchestra.run.smithi177.stdout:899 read 0x7ba058d thru 0x7baf644 (0xf0b8 bytes) 2024-05-08T03:19:39.253 INFO:teuthology.orchestra.run.smithi177.stdout:900 writesame 0x979c451 thru 0x97a46f5 data_size 0x1(0x82a5 bytes) 2024-05-08T03:19:39.264 INFO:teuthology.orchestra.run.smithi177.stdout:901 clone 9 order 22 su 32768 sc 12 2024-05-08T03:19:39.272 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:39.270+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:39.334 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.334 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.342 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:39.363 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:39.362+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:39.464 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:39.462+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:39.486 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.486 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:39.580 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:39.577+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:39.643 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.643 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.649 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:39.665 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:39.664+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:39.766 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:39.764+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:39.795 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.795 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:39.888 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:39.886+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:39.893 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:19:39.948 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone8 from 0xaefb8c8 (overlap 0x209af4) to 0x0 2024-05-08T03:19:39.960 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:39.961 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:39.967 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:39.965+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:39.970 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:40.016 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-05-08T03:19:40.021 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:40.043 INFO:teuthology.orchestra.run.smithi177.stdout:902 read 0x178f6b4 thru 0x179c59b (0xcee8 bytes) 2024-05-08T03:19:40.044 INFO:teuthology.orchestra.run.smithi177.stdout:903 read 0x47bb2c3 thru 0x47bd3e5 (0x2123 bytes) 2024-05-08T03:19:40.044 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xeceef29 2024-05-08T03:19:40.049 INFO:teuthology.orchestra.run.smithi177.stdout:904 trunc from 0xaefb8c8 to 0xeceef29 2024-05-08T03:19:40.060 INFO:teuthology.orchestra.run.smithi177.stdout:905 punch from 0xb390ba0 to 0xb3997cb, (0x8c2b bytes) 2024-05-08T03:19:40.060 INFO:teuthology.orchestra.run.smithi177.stdout:907 read 0x3f7da2b thru 0x3f8ad9a (0xd370 bytes) 2024-05-08T03:19:40.061 INFO:teuthology.orchestra.run.smithi177.stdout:910 read 0x42b415c thru 0x42b68b3 (0x2758 bytes) 2024-05-08T03:19:40.061 INFO:teuthology.orchestra.run.smithi177.stdout:912 read 0xe008b90 thru 0xe00e5c9 (0x5a3a bytes) 2024-05-08T03:19:40.068 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:40.066+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:40.076 INFO:teuthology.orchestra.run.smithi177.stdout:913 trunc from 0xeceef29 to 0x24da82a 2024-05-08T03:19:40.132 INFO:teuthology.orchestra.run.smithi177.stdout:914 punch from 0x112ceb8 to 0x1138524, (0xb66c bytes) 2024-05-08T03:19:40.132 INFO:teuthology.orchestra.run.smithi177.stdout:915 read 0x16f2e1a thru 0x16fcb4b (0x9d32 bytes) 2024-05-08T03:19:40.132 INFO:teuthology.orchestra.run.smithi177.stdout:916 read 0x814f03 thru 0x823006 (0xe104 bytes) 2024-05-08T03:19:40.133 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:40.133 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:40.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:40.146 INFO:teuthology.orchestra.run.smithi177.stdout:918 writesame 0x76ef75a thru 0x76f316c data_size 0x1(0x3a13 bytes) 2024-05-08T03:19:40.156 INFO:teuthology.orchestra.run.smithi177.stdout:919 write 0xa4ffe3e thru 0xa500d6a (0xf2d bytes) 2024-05-08T03:19:40.163 INFO:teuthology.orchestra.run.smithi177.stdout:920 read 0x4641045 thru 0x4644f65 (0x3f21 bytes) 2024-05-08T03:19:40.169 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:40.167+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:40.173 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:40.174 INFO:teuthology.orchestra.run.smithi177.stdout:922 write 0xdf9d4bc thru 0xdfaab67 (0xd6ac bytes) 2024-05-08T03:19:40.174 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:40.182 INFO:teuthology.orchestra.run.smithi177.stdout:925 read 0xc4fded2 thru 0xc50c33a (0xe469 bytes) 2024-05-08T03:19:40.182 INFO:teuthology.orchestra.run.smithi177.stdout:926 read 0x5e34e10 thru 0x5e3b15e (0x634f bytes) 2024-05-08T03:19:40.183 INFO:teuthology.orchestra.run.smithi177.stdout:927 punch from 0x26980b7 to 0x26a55ec, (0xd535 bytes) 2024-05-08T03:19:40.183 INFO:teuthology.orchestra.run.smithi177.stdout:929 punch from 0x4ddd2c to 0x4ea72c, (0xca00 bytes) 2024-05-08T03:19:40.183 INFO:teuthology.orchestra.run.smithi177.stdout:930 read 0xca39d9d thru 0xca42c8a (0x8eee bytes) 2024-05-08T03:19:40.198 INFO:teuthology.orchestra.run.smithi177.stdout:931 trunc from 0xdfaab68 to 0x264125f 2024-05-08T03:19:40.202 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:40.202 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:40.222 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:40.223 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:40.233 INFO:teuthology.orchestra.run.smithi177.stdout:933 write 0xb88babc thru 0xb8911b4 (0x56f9 bytes) 2024-05-08T03:19:40.237 INFO:teuthology.orchestra.run.smithi177.stdout:935 write 0x68fcf46 thru 0x6900070 (0x312b bytes) 2024-05-08T03:19:40.242 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:40.242 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:40.249 INFO:teuthology.orchestra.run.smithi177.stdout:937 trunc from 0xb8911b5 to 0x43f9d6a 2024-05-08T03:19:40.255 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:40.255 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:40.266 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:40.266 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:40.266 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:40.266 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:40.267 INFO:teuthology.orchestra.run.smithi177.stdout:940 read 0x1d0ff47 thru 0x1d13990 (0x3a4a bytes) 2024-05-08T03:19:40.267 INFO:teuthology.orchestra.run.smithi177.stdout:941 write 0x33ffdc6 thru 0x340da66 (0xdca1 bytes) 2024-05-08T03:19:40.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:40.269 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:40.267+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:40.281 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:40.282 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:40.287 INFO:teuthology.orchestra.run.smithi177.stdout:942 write 0xe44d736 thru 0xe458a77 (0xb342 bytes) 2024-05-08T03:19:40.297 INFO:teuthology.orchestra.run.smithi177.stdout:943 write 0x64facee thru 0x6509ad1 (0xede4 bytes) 2024-05-08T03:19:40.301 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:40.301 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:40.303 INFO:teuthology.orchestra.run.smithi177.stdout:944 read 0x7203260 thru 0x7208523 (0x52c4 bytes) 2024-05-08T03:19:40.303 INFO:teuthology.orchestra.run.smithi177.stdout:945 read 0x846abd5 thru 0x8477ec5 (0xd2f1 bytes) 2024-05-08T03:19:40.303 INFO:teuthology.orchestra.run.smithi177.stdout:946 write 0x332f446 thru 0x3338cdf (0x989a bytes) 2024-05-08T03:19:40.308 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:40.314 INFO:teuthology.orchestra.run.smithi177.stdout:948 write 0xea72ecc thru 0xea82556 (0xf68b bytes) 2024-05-08T03:19:40.320 INFO:teuthology.orchestra.run.smithi177.stdout:949 read 0x2606c60 thru 0x260e8a4 (0x7c45 bytes) 2024-05-08T03:19:40.320 INFO:teuthology.orchestra.run.smithi177.stdout:950 clone 10 order 18 su 32768 sc 6 2024-05-08T03:19:40.321 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:40.322 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:40.369 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:40.368+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:40.379 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11826 3.a 3:5299480c:::rbd_data.1182d7df86af.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.6639120380000003, 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.009827967, 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11826 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:40.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713659+0000", 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715527+0000", 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001868562 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715543+0000", 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5341000000000001e-05 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715550+0000", 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4329999999999998e-06 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715605+0000", 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4818999999999998e-05 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715701+0000", 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6454999999999997e-05 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715922+0000", 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00022026700000000001 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.719967+0000", 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040455710000000004 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:40.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723322+0000", 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003355051 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723454+0000", 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013202099999999999 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723485+0000", 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0580000000000002e-05 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12065 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:head [sparse-read 0~5636096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.3637709039999999, 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01338435, 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12065 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013800+0000", 2024-05-08T03:19:40.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.53e-06 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013807+0000", 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5310000000000004e-06 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013825+0000", 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7459999999999999e-05 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013833+0000", 2024-05-08T03:19:40.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013900+0000", 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7475e-05 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013998+0000", 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405000000000007e-05 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.027182+0000", 2024-05-08T03:19:40.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013184695 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.1691495420000004, 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.390 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:19:40.395 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12864 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3801088~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127563+0000", 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2500058379999999, 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083025909999999998, 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.396 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12864 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127563+0000", 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127563+0000", 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127564+0000", 2024-05-08T03:19:40.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7100000000000002e-07 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127566+0000", 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5560000000000002e-06 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127575+0000", 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.037e-06 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3380000000000004e-06 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.135325+0000", 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077446190000000003 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.135410+0000", 2024-05-08T03:19:40.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4756e-05 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.135866+0000", 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045581400000000002 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12877 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127588+0000", 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2499808450000001, 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0089070030000000001, 2024-05-08T03:19:40.400 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12877 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127588+0000", 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127588+0000", 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127588+0000", 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0499999999999999e-07 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127591+0000", 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2110000000000001e-06 2024-05-08T03:19:40.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127598+0000", 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3930000000000002e-06 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127602+0000", 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6109999999999998e-06 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.135892+0000", 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082905570000000005 2024-05-08T03:19:40.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.135940+0000", 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8228e-05 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136495+0000", 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00055459800000000001 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.404 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12878 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 1835008~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.249963046, 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0095219529999999997, 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12878 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7500000000000001e-07 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127608+0000", 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4529999999999999e-06 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127611+0000", 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2610000000000002e-06 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127615+0000", 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.481e-06 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136544+0000", 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0089289710000000008 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136650+0000", 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000105723 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137128+0000", 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000477789 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.408 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12879 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2359296~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2499504990000001, 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011979432999999999, 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12879 2024-05-08T03:19:40.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127619+0000", 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0200000000000001e-07 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127620+0000", 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1659999999999999e-06 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127623+0000", 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0249999999999998e-06 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127625+0000", 2024-05-08T03:19:40.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:19:40.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137175+0000", 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0095499769999999994 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137264+0000", 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9232000000000002e-05 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139598+0000", 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0023335299999999999 2024-05-08T03:19:40.413 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12880 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3932160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2499410580000001, 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012466181999999999, 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.414 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12880 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127629+0000", 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1060000000000001e-06 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127632+0000", 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9569999999999998e-06 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127634+0000", 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3630000000000001e-06 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139646+0000", 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012011164 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139732+0000", 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6537000000000002e-05 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140094+0000", 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00036183500000000002 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12893 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:40.418 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.249525314, 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012522457000000001, 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12893 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.419 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3e-07 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128047+0000", 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8310000000000001e-06 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128058+0000", 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0900000000000001e-05 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128064+0000", 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140135+0000", 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012070166 2024-05-08T03:19:40.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140225+0000", 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0988000000000004e-05 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140566+0000", 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034055799999999997 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.423 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12894 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.249495187, 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013058746, 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12894 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:40.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7800000000000002e-07 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128076+0000", 2024-05-08T03:19:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0899999999999999e-06 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128085+0000", 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3110000000000006e-06 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128091+0000", 2024-05-08T03:19:40.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5989999999999998e-06 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140595+0000", 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012503515999999999 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140642+0000", 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7077999999999999e-05 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141132+0000", 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049067399999999997 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2494690249999998, 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.429 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:19:40.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:19:40.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.433 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2494449539999999, 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.434 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:40.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:19:40.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:19:40.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:19:40.439 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.439 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2494192310000001, 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.440 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:19:40.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:19:40.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.444 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2493875139999999, 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:40.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:19:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:19:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:19:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:19:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:19:40.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:19:40.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:19:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.451 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2493624099999998, 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:19:40.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:19:40.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.249337605, 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.458 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:40.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.465 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2493090609999999, 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:19:40.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:40.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:19:40.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:19:40.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:19:40.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:19:40.472 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.472 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.472 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2492865389999999, 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.473 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.474 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.474 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.474 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.474 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:19:40.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.474 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:40.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:40.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:19:40.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:19:40.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:19:40.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:19:40.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:19:40.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:19:40.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:19:40.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.481 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.481 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:19:40.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:19:40.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.481 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:19:40.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:19:40.482 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.482 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.482 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.249256199, 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.483 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:19:40.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:19:40.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:19:40.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.2492350179999998, 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.489 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.490 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:19:40.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:19:40.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.495 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.495 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.495 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:40.496 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:40.497 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:40.494+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:40.597 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:40.595+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:40.643 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:40.643 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:40.643 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:40.643 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:40.652 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:40.698 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:40.696+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:40.756 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:40.756 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:40.756 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:40.757 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:40.757 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:40.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:40.806 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:40.804+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:40.882 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.0391666009999998, 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.916 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:19:40.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.922 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6972254089999996, 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:19:40.926 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.69721001, 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.927 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:19:40.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:19:40.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6971783680000003, 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:40.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:19:40.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:19:40.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6971650360000003, 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:19:40.935 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:19:40.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.939 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.697155242, 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:40.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:19:40.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6971458290000001, 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:19:40.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.947 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6971363080000001, 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.697126967, 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:19:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6971177520000005, 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6971083900000004, 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970989579999998, 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970895419999996, 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:19:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970805240000004, 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:19:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.697071481, 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:19:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:19:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970591879999999, 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970499209999996, 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:19:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:19:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970360629999997, 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:19:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.6970267699999999, 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:19:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.7040126369999999, 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:19:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:41.001 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:41.002 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:40.940+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:41.003 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone9 from 0xea82557 (overlap 0x24da82a) to 0x126d415 2024-05-08T03:19:41.003 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:41.001+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:41.004 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:41.060 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-05-08T03:19:41.061 INFO:teuthology.orchestra.run.smithi177.stdout:952 read 0x403498 thru 0x4105ef (0xd158 bytes) 2024-05-08T03:19:41.070 INFO:teuthology.orchestra.run.smithi177.stdout:956 writesame 0xea8cd56 thru 0xea93335 data_size 0x4(0x65e0 bytes) 2024-05-08T03:19:41.074 INFO:teuthology.orchestra.run.smithi177.stdout:957 write 0x39829d9 thru 0x398e8e3 (0xbf0b bytes) 2024-05-08T03:19:41.084 INFO:teuthology.orchestra.run.smithi177.stdout:961 trunc from 0xea93336 to 0xa3f924c 2024-05-08T03:19:41.103 INFO:teuthology.orchestra.run.smithi177.stdout:962 punch from 0x5b89a44 to 0x5b910fd, (0x76b9 bytes) 2024-05-08T03:19:41.104 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:41.102+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:41.113 INFO:teuthology.orchestra.run.smithi177.stdout:963 write 0xe474080 thru 0xe478b48 (0x4ac9 bytes) 2024-05-08T03:19:41.120 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:41.120 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:41.120 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:41.120 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:41.122 INFO:teuthology.orchestra.run.smithi177.stdout:965 write 0x81eb4a6 thru 0x81ed261 (0x1dbc bytes) 2024-05-08T03:19:41.125 INFO:teuthology.orchestra.run.smithi177.stdout:966 read 0xce9993 thru 0xcf7548 (0xdbb6 bytes) 2024-05-08T03:19:41.125 INFO:teuthology.orchestra.run.smithi177.stdout:967 punch from 0xdf3005c to 0xdf36ba0, (0x6b44 bytes) 2024-05-08T03:19:41.125 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:41.125 INFO:teuthology.orchestra.run.smithi177.stdout:969 writesame 0x8652f28 thru 0x865db34 data_size 0x17(0xac0d bytes) 2024-05-08T03:19:41.130 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:41.131 INFO:teuthology.orchestra.run.smithi177.stdout:970 write 0x853890b thru 0x853cca7 (0x439d bytes) 2024-05-08T03:19:41.139 INFO:teuthology.orchestra.run.smithi177.stdout:971 trunc from 0xe478b49 to 0x9d294a4 2024-05-08T03:19:41.176 INFO:teuthology.orchestra.run.smithi177.stdout:972 trunc from 0x9d294a4 to 0x5bce45b 2024-05-08T03:19:41.200 INFO:teuthology.orchestra.run.smithi177.stdout:973 read 0x34a8cd1 thru 0x34a9d1c (0x104c bytes) 2024-05-08T03:19:41.204 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:41.203+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:41.211 INFO:teuthology.orchestra.run.smithi177.stdout:974 write 0x96a307b thru 0x96aaed9 (0x7e5f bytes) 2024-05-08T03:19:41.220 INFO:teuthology.orchestra.run.smithi177.stdout:976 trunc from 0x96aaeda to 0x6fd3c78 2024-05-08T03:19:41.229 INFO:teuthology.orchestra.run.smithi177.stdout:977 punch from 0x2f7e474 to 0x2f838ab, (0x5437 bytes) 2024-05-08T03:19:41.229 INFO:teuthology.orchestra.run.smithi177.stdout:978 trunc from 0x6fd3c78 to 0x69dacd4 2024-05-08T03:19:41.234 INFO:teuthology.orchestra.run.smithi177.stdout:979 read 0x530b0d2 thru 0x5318dc1 (0xdcf0 bytes) 2024-05-08T03:19:41.244 INFO:teuthology.orchestra.run.smithi177.stdout:980 trunc from 0x69dacd4 to 0xe56bb39 2024-05-08T03:19:41.249 INFO:teuthology.orchestra.run.smithi177.stdout:981 read 0xca6669c thru 0xca74ced (0xe652 bytes) 2024-05-08T03:19:41.249 INFO:teuthology.orchestra.run.smithi177.stdout:983 read 0x4416f92 thru 0x4422231 (0xb2a0 bytes) 2024-05-08T03:19:41.249 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:41.249 INFO:teuthology.orchestra.run.smithi177.stdout:985 clone 11 order 21 su 16384 sc 3 2024-05-08T03:19:41.253 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:41.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:41.253 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:41.253 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:41.253 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:41.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:41.306 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:41.304+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:41.405 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:41.404+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:41.409 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:41.409 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:41.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:41.409 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:41.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.207259762, 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:19:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:19:41.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.414 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.207250311999999, 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.415 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:41.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:19:41.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.207237576000001, 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:19:41.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:19:41.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.424 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.207227, 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.425 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:19:41.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.207217612999999, 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.430 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.207208433, 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.435 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976383397999999, 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:19:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976351161, 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:19:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:19:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976337831, 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:19:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976322857, 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:19:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:19:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976311548, 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:19:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976293879, 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:19:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976278681, 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:19:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:19:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.976264586999999, 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:19:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:19:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.2013106139999996, 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:19:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:19:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.2784168119999997, 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:19:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:19:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.2784033240000001, 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:19:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.2783743430000003, 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.2783477369999998, 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:19:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:19:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:19:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.8455994759999999, 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:19:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:19:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:19:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:19:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:19:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:19:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:19:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:19:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:19:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:19:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:19:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:19:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:41.520 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:41.522 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:41.522 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:41.519+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:41.622 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:41.621+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:41.667 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:41.667 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:41.667 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:41.667 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:41.675 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:41.724 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:41.722+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:41.820 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:41.820 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:41.820 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:41.820 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:41.820 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:41.824 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:41.823+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:41.829 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:41.925 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:41.924+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:41.969 INFO:teuthology.orchestra.run.smithi177.stdout:flattening image image_client.0-clone10 2024-05-08T03:19:42.002 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:115 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'65)", 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.915035792000001, 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014688414, 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083947+0000", 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1900000000000001e-07 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083985+0000", 2024-05-08T03:19:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7769999999999999e-05 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084001+0000", 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.592e-05 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084009+0000", 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9109999999999993e-06 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084686+0000", 2024-05-08T03:19:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00067690100000000004 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084703+0000", 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.755e-05 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098569+0000", 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013865446999999999 2024-05-08T03:19:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098635+0000", 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6095999999999999e-05 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:116 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'65)", 2024-05-08T03:19:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.914842922999998, 2024-05-08T03:19:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014525575000000001, 2024-05-08T03:19:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084140+0000", 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1900000000000003e-07 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084144+0000", 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6149999999999999e-06 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084153+0000", 2024-05-08T03:19:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6459999999999999e-06 2024-05-08T03:19:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084159+0000", 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0079999999999999e-06 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085434+0000", 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001275263 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085455+0000", 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0893000000000001e-05 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098638+0000", 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013182523 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098665+0000", 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7107999999999999e-05 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:117 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'65)", 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.914174876000001, 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013907645, 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084808+0000", 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-07 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084813+0000", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2259999999999996e-06 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084822+0000", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5830000000000007e-06 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084830+0000", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9650000000000008e-06 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085897+0000", 2024-05-08T03:19:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001067496 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085917+0000", 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0080000000000001e-05 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098667+0000", 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012749462 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098715+0000", 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8285999999999998e-05 2024-05-08T03:19:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:824 3.1a e25/19 3:5e68eb20:::rbd_object_map.11821677beed:head v 25'15, mlcod=25'6)", 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:27.875362+0000", 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.123619986, 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092165530000000006, 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875362+0000", 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875362+0000", 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875363+0000", 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875370+0000", 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0849999999999999e-06 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875378+0000", 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7149999999999998e-06 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875383+0000", 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0420000000000002e-06 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875425+0000", 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2568999999999998e-05 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875437+0000", 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1902000000000001e-05 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.884540+0000", 2024-05-08T03:19:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0091033910000000006 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.884579+0000", 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8408000000000002e-05 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2079 3.19 3:99a2a0a7:::rbd_data.1182c698972d.0000000000000310:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.500202+0000", 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.498780556, 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089751959999999995, 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2079 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500202+0000", 2024-05-08T03:19:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500202+0000", 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500203+0000", 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3349999999999999e-06 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500210+0000", 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6950000000000003e-06 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500223+0000", 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3388e-05 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500229+0000", 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7150000000000003e-06 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500279+0000", 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9855000000000003e-05 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500358+0000", 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9396999999999999e-05 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.501764+0000", 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0014062499999999999 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509058+0000", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0072933130000000001 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509075+0000", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7394e-05 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509129+0000", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3653999999999998e-05 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509177+0000", 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8199999999999999e-05 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11618 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.693988+0000", 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3049940849999997, 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092573900000000008, 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11618 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693988+0000", 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693988+0000", 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693989+0000", 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5699999999999998e-07 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693990+0000", 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.734e-06 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694003+0000", 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3064999999999999e-05 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694008+0000", 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.843e-06 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.702853+0000", 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0088450109999999998 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.702912+0000", 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8352e-05 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703246+0000", 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033402799999999999 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11619 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.304967768, 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097089580000000002, 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11619 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.84e-07 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694017+0000", 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8160000000000001e-06 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694023+0000", 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:19:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694030+0000", 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5320000000000003e-06 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703325+0000", 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092952739999999992 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703406+0000", 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1277000000000001e-05 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703723+0000", 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031717600000000001 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11620 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3049362950000001, 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010112856, 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11620 2024-05-08T03:19:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.51e-07 2024-05-08T03:19:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694049+0000", 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4660000000000002e-06 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694058+0000", 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3300000000000005e-06 2024-05-08T03:19:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694063+0000", 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8690000000000003e-06 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703752+0000", 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0096887310000000008 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703799+0000", 2024-05-08T03:19:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7342999999999997e-05 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704159+0000", 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00035966600000000002 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11621 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694069+0000", 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3049129859999997, 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010542635999999999, 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11621 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694069+0000", 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694069+0000", 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694070+0000", 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5100000000000001e-07 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694072+0000", 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8950000000000001e-06 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694083+0000", 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.152e-05 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694110+0000", 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7044000000000001e-05 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704203+0000", 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010093365999999999 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704282+0000", 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8053000000000006e-05 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704612+0000", 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033040700000000001 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11622 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3048456699999997, 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010835354, 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11622 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5899999999999996e-07 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694140+0000", 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9629999999999998e-06 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694148+0000", 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1599999999999998e-06 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694156+0000", 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0050000000000004e-06 2024-05-08T03:19:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704651+0000", 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010494852000000001 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704697+0000", 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6140999999999998e-05 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704972+0000", 2024-05-08T03:19:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027457399999999998 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11623 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3048167199999998, 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011222263, 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11623 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3300000000000003e-07 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694168+0000", 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4480000000000001e-06 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694189+0000", 2024-05-08T03:19:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0080000000000001e-05 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694194+0000", 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6790000000000002e-06 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705009+0000", 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010814361999999999 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705073+0000", 2024-05-08T03:19:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4463999999999998e-05 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705388+0000", 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031479700000000002 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11624 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.304780482, 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011578634000000001, 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11624 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3499999999999996e-07 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694206+0000", 2024-05-08T03:19:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9689999999999996e-06 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694212+0000", 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1770000000000003e-06 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694217+0000", 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.87e-06 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705425+0000", 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011207590999999999 2024-05-08T03:19:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705478+0000", 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2787000000000001e-05 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705780+0000", 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030270500000000001 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11625 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3047587619999996, 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011994640000000001, 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11625 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3300000000000003e-07 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694226+0000", 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2859999999999998e-06 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694232+0000", 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8610000000000003e-06 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694237+0000", 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4120000000000003e-06 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705824+0000", 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011587123 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705900+0000", 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5887e-05 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706218+0000", 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031863799999999999 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11626 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3047397509999996, 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012453031, 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11626 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.08e-07 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694245+0000", 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.142e-06 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694251+0000", 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.852e-06 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694256+0000", 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6360000000000002e-06 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706248+0000", 2024-05-08T03:19:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011992829999999999 2024-05-08T03:19:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706336+0000", 2024-05-08T03:19:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8077000000000004e-05 2024-05-08T03:19:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706696+0000", 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000359086 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11627 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3047213610000004, 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01280324, 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11627 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999999e-07 2024-05-08T03:19:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694263+0000", 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9369999999999998e-06 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694269+0000", 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6280000000000002e-06 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694277+0000", 2024-05-08T03:19:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1799999999999996e-06 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706742+0000", 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012465285 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706809+0000", 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6917999999999999e-05 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707064+0000", 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025493199999999998 2024-05-08T03:19:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11628 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3046813459999997, 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013149357, 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11628 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4499999999999997e-07 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694304+0000", 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7379999999999999e-06 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694310+0000", 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2519999999999996e-06 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694315+0000", 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6009999999999997e-06 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707093+0000", 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012777726 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707151+0000", 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8097999999999998e-05 2024-05-08T03:19:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707450+0000", 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00029939700000000002 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12074 3.4 3:216c60a0:::rbd_data.11824ee2abc9.000000000000000a:head [sparse-read 0~5505024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.013854+0000", 2024-05-08T03:19:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.9851285000000001, 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010655777999999999, 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12074 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013854+0000", 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013854+0000", 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013857+0000", 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0730000000000001e-06 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013866+0000", 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6840000000000002e-06 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013988+0000", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012273300000000001 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014001+0000", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2505e-05 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014083+0000", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2636e-05 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014170+0000", 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6953999999999995e-05 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.024510+0000", 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010339193 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12075 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:head [sparse-read 0~5505024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.9849536959999998, 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017019135000000001, 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12075 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0200000000000002e-07 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014033+0000", 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-06 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014045+0000", 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1929e-05 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014052+0000", 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5969999999999997e-06 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.024570+0000", 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010517888 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.024681+0000", 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011136799999999999 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.031048+0000", 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.006366521 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13005 3.0 3:01f49529:::rbd_data.118231ecabb9.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.7936769420000003, 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.021952854000000001, 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13005 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:19:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205306+0000", 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0999999999999997e-07 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205310+0000", 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8249999999999998e-06 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205319+0000", 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6500000000000002e-06 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205325+0000", 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3060000000000003e-06 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205368+0000", 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2809000000000003e-05 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205418+0000", 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9920999999999999e-05 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.209135+0000", 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037170160000000001 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227134+0000", 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017998962 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227171+0000", 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7462999999999997e-05 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227209+0000", 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8275999999999997e-05 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227258+0000", 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8816000000000002e-05 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13450 3.0 3:06f1dff9:::rbd_data.118231ecabb9.0000000000000068:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.1580556550000001, 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015476551, 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13450 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7899999999999997e-07 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840932+0000", 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.95e-06 2024-05-08T03:19:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840943+0000", 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840949+0000", 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5589999999999996e-06 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840998+0000", 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9718000000000001e-05 2024-05-08T03:19:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841058+0000", 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0000999999999997e-05 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.843397+0000", 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0023384510000000001 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856224+0000", 2024-05-08T03:19:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012827668 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856283+0000", 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8263000000000001e-05 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856350+0000", 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7577999999999996e-05 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856403+0000", 2024-05-08T03:19:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.295e-05 2024-05-08T03:19:42.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.103 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:42.103 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:42.103 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:42.104 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:42.105 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:42.102+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:42.185 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-05-08T03:19:42.205 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:42.204+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:42.211 INFO:teuthology.orchestra.run.smithi177.stdout:986 write 0xcd2298b thru 0xcd2f73a (0xcdb0 bytes) 2024-05-08T03:19:42.224 INFO:teuthology.orchestra.run.smithi177.stdout:988 read 0xb985ea1 thru 0xb994c51 (0xedb1 bytes) 2024-05-08T03:19:42.224 INFO:teuthology.orchestra.run.smithi177.stdout:989 read 0x2c9227 thru 0x2d4564 (0xb33e bytes) 2024-05-08T03:19:42.224 INFO:teuthology.orchestra.run.smithi177.stdout:990 read 0x641c72e thru 0x6424b91 (0x8464 bytes) 2024-05-08T03:19:42.224 INFO:teuthology.orchestra.run.smithi177.stdout:991 read 0x98de7b5 thru 0x98e2720 (0x3f6c bytes) 2024-05-08T03:19:42.224 INFO:teuthology.orchestra.run.smithi177.stdout:992 write 0x5cdfaa3 thru 0x5ce6b89 (0x70e7 bytes) 2024-05-08T03:19:42.232 INFO:teuthology.orchestra.run.smithi177.stdout:993 write 0x39029e0 thru 0x390d1f1 (0xa812 bytes) 2024-05-08T03:19:42.240 INFO:teuthology.orchestra.run.smithi177.stdout:994 punch from 0x25a583e to 0x25a6dcf, (0x1591 bytes) 2024-05-08T03:19:42.240 INFO:teuthology.orchestra.run.smithi177.stdout:996 read 0x6d67680 thru 0x6d70ff1 (0x9972 bytes) 2024-05-08T03:19:42.240 INFO:teuthology.orchestra.run.smithi177.stdout:997 writesame 0xaa7c37b thru 0xaa817dc data_size 0x7(0x5462 bytes) 2024-05-08T03:19:42.245 INFO:teuthology.orchestra.run.smithi177.stdout:999 write 0x937b882 thru 0x9381e44 (0x65c3 bytes) 2024-05-08T03:19:42.256 INFO:teuthology.orchestra.run.smithi177.stdout:1001 write 0xdd41a84 thru 0xdd465a3 (0x4b20 bytes) 2024-05-08T03:19:42.262 INFO:teuthology.orchestra.run.smithi177.stdout:1002 read 0x9f04b01 thru 0x9f0c13b (0x763b bytes) 2024-05-08T03:19:42.266 INFO:teuthology.orchestra.run.smithi177.stdout:1003 trunc from 0xe56bb39 to 0xae2b476 2024-05-08T03:19:42.267 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:42.267 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:42.267 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:42.267 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:42.277 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:42.293 INFO:teuthology.orchestra.run.smithi177.stdout:1004 write 0x6709607 thru 0x671300c (0x9a06 bytes) 2024-05-08T03:19:42.306 INFO:teuthology.orchestra.run.smithi177.stdout:1005 writesame 0x3d7453a thru 0x3d7ac2f data_size 0x1(0x66f6 bytes) 2024-05-08T03:19:42.307 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:42.305+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:42.318 INFO:teuthology.orchestra.run.smithi177.stdout:1007 read 0x5cbb2ce thru 0x5cc98c8 (0xe5fb bytes) 2024-05-08T03:19:42.328 INFO:teuthology.orchestra.run.smithi177.stdout:1008 writesame 0xe9f0775 thru 0xe9fa315 data_size 0x1(0x9ba1 bytes) 2024-05-08T03:19:42.328 INFO:teuthology.orchestra.run.smithi177.stdout:1010 punch from 0x2992b7a to 0x2995e40, (0x32c6 bytes) 2024-05-08T03:19:42.328 INFO:teuthology.orchestra.run.smithi177.stdout:1013 write 0xb16b5e1 thru 0xb172d69 (0x7789 bytes) 2024-05-08T03:19:42.329 INFO:teuthology.orchestra.run.smithi177.stdout:1014 punch from 0xbc2ac50 to 0xbc39fad, (0xf35d bytes) 2024-05-08T03:19:42.329 INFO:teuthology.orchestra.run.smithi177.stdout:1015 write 0x291fc6f thru 0x29230b9 (0x344b bytes) 2024-05-08T03:19:42.333 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:42.337 INFO:teuthology.orchestra.run.smithi177.stdout:1016 punch from 0x1349259 to 0x13546f1, (0xb498 bytes) 2024-05-08T03:19:42.337 INFO:teuthology.orchestra.run.smithi177.stdout:1017 write 0x92357ec thru 0x9239241 (0x3a56 bytes) 2024-05-08T03:19:42.356 INFO:teuthology.orchestra.run.smithi177.stdout:1018 trunc from 0xe9fa316 to 0x224658d 2024-05-08T03:19:42.392 INFO:teuthology.orchestra.run.smithi177.stdout:1020 read 0x22283fe thru 0x22378f2 (0xf4f5 bytes) 2024-05-08T03:19:42.407 INFO:teuthology.orchestra.run.smithi177.stdout:1021 writesame 0x993890d thru 0x993dce8 data_size 0x6(0x53dc bytes) 2024-05-08T03:19:42.407 INFO:teuthology.orchestra.run.smithi177.stdout:1022 read 0x37ed4f7 thru 0x37f0131 (0x2c3b bytes) 2024-05-08T03:19:42.407 INFO:teuthology.orchestra.run.smithi177.stdout:1023 punch from 0x82e107 to 0x838a3d, (0xa936 bytes) 2024-05-08T03:19:42.407 INFO:teuthology.orchestra.run.smithi177.stdout:1026 write 0x17cfca thru 0x1819b7 (0x49ee bytes) 2024-05-08T03:19:42.408 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:42.405+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:42.415 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.418 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:42.418 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:42.418 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:42.418 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:42.418 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:42.428 INFO:teuthology.orchestra.run.smithi177.stdout:1028 write 0xea034e4 thru 0xea069c2 (0x34df bytes) 2024-05-08T03:19:42.431 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:42.447 INFO:teuthology.orchestra.run.smithi177.stdout:1030 trunc from 0xea069c3 to 0x25cd31d 2024-05-08T03:19:42.458 INFO:teuthology.orchestra.run.smithi177.stdout:1031 trunc from 0x25cd31d to 0x322bf26 2024-05-08T03:19:42.464 INFO:teuthology.orchestra.run.smithi177.stdout:1032 punch from 0x5c501e to 0x5c63cd, (0x13af bytes) 2024-05-08T03:19:42.464 INFO:teuthology.orchestra.run.smithi177.stdout:1033 punch from 0x8b079 to 0x97f37, (0xcebe bytes) 2024-05-08T03:19:42.465 INFO:teuthology.orchestra.run.smithi177.stdout:1034 write 0x2eae2e0 thru 0x2eb27f7 (0x4518 bytes) 2024-05-08T03:19:42.468 INFO:teuthology.orchestra.run.smithi177.stdout:1036 write 0x19e9a8e thru 0x19f393a (0x9ead bytes) 2024-05-08T03:19:42.474 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.475 INFO:teuthology.orchestra.run.smithi177.stdout:1039 read 0xe6b739 thru 0xe6c8af (0x1177 bytes) 2024-05-08T03:19:42.490 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:42.490 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:42.493 INFO:teuthology.orchestra.run.smithi177.stdout:1041 writesame 0xc7c9b3a thru 0xc7ce0ca data_size 0x1(0x4591 bytes) 2024-05-08T03:19:42.493 INFO:teuthology.orchestra.run.smithi177.stdout:1043 punch from 0xabbda74 to 0xabc15f6, (0x3b82 bytes) 2024-05-08T03:19:42.493 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.505 INFO:teuthology.orchestra.run.smithi177.stdout:1047 trunc from 0xc7ce0cb to 0x235ee1d 2024-05-08T03:19:42.508 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:42.507+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:42.509 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:42.509 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:42.518 INFO:teuthology.orchestra.run.smithi177.stdout:1049 read 0x1487d75 thru 0x149288b (0xab17 bytes) 2024-05-08T03:19:42.518 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.518 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.523 INFO:teuthology.orchestra.run.smithi177.stdout:1052 trunc from 0x235ee1d to 0x587f55 2024-05-08T03:19:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:42.531 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:42.534 INFO:teuthology.orchestra.run.smithi177.stdout:1053 read 0x9de9b thru 0x9ef73 (0x10d9 bytes) 2024-05-08T03:19:42.550 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:42.550 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:42.552 INFO:teuthology.orchestra.run.smithi177.stdout:1054 writesame 0xa1ffb42 thru 0xa206c1d data_size 0x4(0x70dc bytes) 2024-05-08T03:19:42.560 INFO:teuthology.orchestra.run.smithi177.stdout:1055 write 0xc61f22e thru 0xc621372 (0x2145 bytes) 2024-05-08T03:19:42.564 INFO:teuthology.orchestra.run.smithi177.stdout:1056 write 0x36e7d84 thru 0x36efd92 (0x800f bytes) 2024-05-08T03:19:42.571 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:42.571 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:42.575 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:42.576 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:42.576 INFO:teuthology.orchestra.run.smithi177.stdout:1057 write 0xe1c9dee thru 0xe1cd72a (0x393d bytes) 2024-05-08T03:19:42.585 INFO:teuthology.orchestra.run.smithi177.stdout:1058 write 0xc17cb6f thru 0xc188df0 (0xc282 bytes) 2024-05-08T03:19:42.586 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:42.591 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:42.591 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:42.597 INFO:teuthology.orchestra.run.smithi177.stdout:1060 read 0x8d15d22 thru 0x8d2069c (0xa97b bytes) 2024-05-08T03:19:42.597 INFO:teuthology.orchestra.run.smithi177.stdout:1061 write 0xb9f5fa9 thru 0xba01c98 (0xbcf0 bytes) 2024-05-08T03:19:42.605 INFO:teuthology.orchestra.run.smithi177.stdout:1062 punch from 0x9b2dbe1 to 0x9b395c0, (0xb9df bytes) 2024-05-08T03:19:42.605 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.606 INFO:teuthology.orchestra.run.smithi177.stdout:1064 read 0x4ac39e2 thru 0x4acc513 (0x8b32 bytes) 2024-05-08T03:19:42.606 INFO:teuthology.orchestra.run.smithi177.stdout:1066 read 0x5e12ce1 thru 0x5e1f1c7 (0xc4e7 bytes) 2024-05-08T03:19:42.606 INFO:teuthology.orchestra.run.smithi177.stdout:1067 write 0xda412f7 thru 0xda448d0 (0x35da bytes) 2024-05-08T03:19:42.609 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:42.607+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:42.610 INFO:teuthology.orchestra.run.smithi177.stdout:1068 punch from 0x73e31c9 to 0x73e7ea6, (0x4cdd bytes) 2024-05-08T03:19:42.610 INFO:teuthology.orchestra.run.smithi177.stdout:1069 punch from 0x33e33a3 to 0x33e33ca, (0x27 bytes) 2024-05-08T03:19:42.610 INFO:teuthology.orchestra.run.smithi177.stdout:1070 write 0x6701102 thru 0x670a7c2 (0x96c1 bytes) 2024-05-08T03:19:42.614 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:42.614 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:42.618 INFO:teuthology.orchestra.run.smithi177.stdout:1071 punch from 0x2e39615 to 0x2e485c4, (0xefaf bytes) 2024-05-08T03:19:42.618 INFO:teuthology.orchestra.run.smithi177.stdout:1072 write 0x623c78a thru 0x623c8e4 (0x15b bytes) 2024-05-08T03:19:42.622 INFO:teuthology.orchestra.run.smithi177.stdout:1076 read 0xd72debb thru 0xd739d5d (0xbea3 bytes) 2024-05-08T03:19:42.622 INFO:teuthology.orchestra.run.smithi177.stdout:1077 read 0x183d2e2 thru 0x184957b (0xc29a bytes) 2024-05-08T03:19:42.622 INFO:teuthology.orchestra.run.smithi177.stdout:1078 writesame 0xc38cee0 thru 0xc39c113 data_size 0x6(0xf234 bytes) 2024-05-08T03:19:42.630 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.631 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.631 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.631 INFO:teuthology.orchestra.run.smithi177.stdout:1084 write 0x734ee97 thru 0x7353eff (0x5069 bytes) 2024-05-08T03:19:42.635 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:42.635 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:42.636 INFO:teuthology.orchestra.run.smithi177.stdout:1085 write 0x3c2f5fe thru 0x3c3028d (0xc90 bytes) 2024-05-08T03:19:42.641 INFO:teuthology.orchestra.run.smithi177.stdout:1087 writesame 0xa1087e2 thru 0xa1160aa data_size 0xd(0xd8c9 bytes) 2024-05-08T03:19:42.650 INFO:teuthology.orchestra.run.smithi177.stdout:1088 punch from 0x548d4b to 0x54ab77, (0x1e2c bytes) 2024-05-08T03:19:42.652 INFO:teuthology.orchestra.run.smithi177.stdout:1089 trunc from 0xe1cd72b to 0xc38af57 2024-05-08T03:19:42.668 INFO:teuthology.orchestra.run.smithi177.stdout:1090 writesame 0x4136f1e thru 0x4137e06 data_size 0x1(0xee9 bytes) 2024-05-08T03:19:42.675 INFO:teuthology.orchestra.run.smithi177.stdout:1091 write 0xcb8d7e5 thru 0xcb9a2a1 (0xcabd bytes) 2024-05-08T03:19:42.685 INFO:teuthology.orchestra.run.smithi177.stdout:1092 writesame 0xa8f61ec thru 0xa905d6b data_size 0x40(0xfb80 bytes) 2024-05-08T03:19:42.692 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.693 INFO:teuthology.orchestra.run.smithi177.stdout:1094 write 0x3e3e302 thru 0x3e3e899 (0x598 bytes) 2024-05-08T03:19:42.695 INFO:teuthology.orchestra.run.smithi177.stdout:1095 write 0x9d0f4b0 thru 0x9d1cba3 (0xd6f4 bytes) 2024-05-08T03:19:42.698 INFO:teuthology.orchestra.run.smithi177.stdout:1096 write 0xb8f3b3 thru 0xb9a930 (0xb57e bytes) 2024-05-08T03:19:42.703 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.703 INFO:teuthology.orchestra.run.smithi177.stdout:1098 punch from 0x52e9bd9 to 0x52f1d38, (0x815f bytes) 2024-05-08T03:19:42.703 INFO:teuthology.orchestra.run.smithi177.stdout:1101 punch from 0x17b1dcb to 0x17b5e24, (0x4059 bytes) 2024-05-08T03:19:42.703 INFO:teuthology.orchestra.run.smithi177.stdout:1102 write 0x3faffee thru 0x3fb0929 (0x93c bytes) 2024-05-08T03:19:42.705 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.710 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:42.708+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:42.715 INFO:teuthology.orchestra.run.smithi177.stdout:1104 trunc from 0xcb9a2a2 to 0x4803330 2024-05-08T03:19:42.720 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:42.720 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:42.729 INFO:teuthology.orchestra.run.smithi177.stdout:1105 write 0x14ab5eb thru 0x14b38de (0x82f4 bytes) 2024-05-08T03:19:42.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:42.745 INFO:teuthology.orchestra.run.smithi177.stdout:1107 trunc from 0x4803330 to 0xe50911a 2024-05-08T03:19:42.751 INFO:teuthology.orchestra.run.smithi177.stdout:1109 write 0xcd409b7 thru 0xcd4efae (0xe5f8 bytes) 2024-05-08T03:19:42.755 INFO:teuthology.orchestra.run.smithi177.stdout:1110 read 0x4019760 thru 0x40268ce (0xd16f bytes) 2024-05-08T03:19:42.756 INFO:teuthology.orchestra.run.smithi177.stdout:1112 read 0x92fdaf thru 0x937c85 (0x7ed7 bytes) 2024-05-08T03:19:42.756 INFO:teuthology.orchestra.run.smithi177.stdout:1113 punch from 0x9199d5e to 0x919c364, (0x2606 bytes) 2024-05-08T03:19:42.756 INFO:teuthology.orchestra.run.smithi177.stdout:1114 read 0x13e279c thru 0x13e8247 (0x5aac bytes) 2024-05-08T03:19:42.757 INFO:teuthology.orchestra.run.smithi177.stdout:1115 write 0x2468667 thru 0x2474799 (0xc133 bytes) 2024-05-08T03:19:42.765 INFO:teuthology.orchestra.run.smithi177.stdout:1116 read 0xa261024 thru 0xa265e61 (0x4e3e bytes) 2024-05-08T03:19:42.771 INFO:teuthology.orchestra.run.smithi177.stdout:1117 write 0xea45bdc thru 0xea4dbf7 (0x801c bytes) 2024-05-08T03:19:42.776 INFO:teuthology.orchestra.run.smithi177.stdout:1118 write 0x4f047e3 thru 0x4f123df (0xdbfd bytes) 2024-05-08T03:19:42.787 INFO:teuthology.orchestra.run.smithi177.stdout:1121 read 0x5148ed9 thru 0x5154ead (0xbfd5 bytes) 2024-05-08T03:19:42.788 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.788 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.788 INFO:teuthology.orchestra.run.smithi177.stdout:1127 write 0x3064f75 thru 0x3067671 (0x26fd bytes) 2024-05-08T03:19:42.790 INFO:teuthology.orchestra.run.smithi177.stdout:1128 write 0x44b8556 thru 0x44bba9c (0x3547 bytes) 2024-05-08T03:19:42.793 INFO:teuthology.orchestra.run.smithi177.stdout:1130 read 0x30b72d1 thru 0x30c60e8 (0xee18 bytes) 2024-05-08T03:19:42.794 INFO:teuthology.orchestra.run.smithi177.stdout:1131 write 0x4da28e1 thru 0x4db15f1 (0xed11 bytes) 2024-05-08T03:19:42.804 INFO:teuthology.orchestra.run.smithi177.stdout:1132 writesame 0x910e262 thru 0x9110ad5 data_size 0x6(0x2874 bytes) 2024-05-08T03:19:42.810 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:42.808+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:42.816 INFO:teuthology.orchestra.run.smithi177.stdout:1134 trunc from 0xea4dbf8 to 0x81cda8e 2024-05-08T03:19:42.830 INFO:teuthology.orchestra.run.smithi177.stdout:1135 read 0x24591fb thru 0x245c858 (0x365e bytes) 2024-05-08T03:19:42.830 INFO:teuthology.orchestra.run.smithi177.stdout:1136 read 0x1d5bb0b thru 0x1d6a80f (0xed05 bytes) 2024-05-08T03:19:42.840 INFO:teuthology.orchestra.run.smithi177.stdout:1137 write 0x9f11160 thru 0x9f1a591 (0x9432 bytes) 2024-05-08T03:19:42.848 INFO:teuthology.orchestra.run.smithi177.stdout:1138 read 0x22a9d02 thru 0x22ad8f3 (0x3bf2 bytes) 2024-05-08T03:19:42.864 INFO:teuthology.orchestra.run.smithi177.stdout:1141 write 0xd47f0cb thru 0xd4884f2 (0x9428 bytes) 2024-05-08T03:19:42.866 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:42.866 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:42.872 INFO:teuthology.orchestra.run.smithi177.stdout:1143 write 0x69acf0a thru 0x69af698 (0x278f bytes) 2024-05-08T03:19:42.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:42.877 INFO:teuthology.orchestra.run.smithi177.stdout:1144 write 0x992899d thru 0x9937ad4 (0xf138 bytes) 2024-05-08T03:19:42.878 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:42.878 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:42.878 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 5, 6, 7] 2024-05-08T03:19:42.916 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-05-08T03:19:42.917 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1145 write 0x44b8d65 thru 0x44bd0f4 (0x4390 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1147 write 0x4fca1cb thru 0x4fcab1d (0x953 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1148 read 0x593ff22 thru 0x59482c3 (0x83a2 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1149 flatten 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1150 read 0x8db538e thru 0x8db6627 (0x129a bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1153 read 0x925f21 thru 0x92827b (0x235b bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1154 read 0x23a4128 thru 0x23b1428 (0xd301 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1156 read 0x7ab1a11 thru 0x7aba3c1 (0x89b1 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1157 read 0xd347166 thru 0xd34c7b7 (0x5652 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1159 read 0x96e335 thru 0x97726a (0x8f36 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1160 write 0xa1b47ee thru 0xa1b6852 (0x2065 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1161 read 0x294d71f thru 0x29551f3 (0x7ad5 bytes) 2024-05-08T03:19:42.921 INFO:teuthology.orchestra.run.smithi177.stdout:1162 punch from 0xc3b3e04 to 0xc3bedc8, (0xafc4 bytes) 2024-05-08T03:19:42.922 INFO:teuthology.orchestra.run.smithi177.stdout:1163 read 0x76d9e8c thru 0x76dca57 (0x2bcc bytes) 2024-05-08T03:19:42.922 INFO:teuthology.orchestra.run.smithi177.stdout:1164 read 0x6a4fda6 thru 0x6a5c77e (0xc9d9 bytes) 2024-05-08T03:19:42.922 INFO:teuthology.orchestra.run.smithi177.stdout:1165 write 0x1593be8 thru 0x1597f47 (0x4360 bytes) 2024-05-08T03:19:42.922 INFO:teuthology.orchestra.run.smithi177.stdout:1166 writesame 0x3227a0e thru 0x322d8ab data_size 0x21(0x5e9e bytes) 2024-05-08T03:19:42.922 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.923 INFO:teuthology.orchestra.run.smithi177.stdout:1170 read 0x57473f9 thru 0x57489ec (0x15f4 bytes) 2024-05-08T03:19:42.923 INFO:teuthology.orchestra.run.smithi177.stdout:1171 read 0x51a9bc0 thru 0x51aabe2 (0x1023 bytes) 2024-05-08T03:19:42.923 INFO:teuthology.orchestra.run.smithi177.stdout:1172 read 0x7282e2d thru 0x728cebb (0xa08f bytes) 2024-05-08T03:19:42.923 INFO:teuthology.orchestra.run.smithi177.stdout:1173 trunc from 0xd4884f3 to 0xd56cc95 2024-05-08T03:19:42.923 INFO:teuthology.orchestra.run.smithi177.stdout:1174 write 0x1710221 thru 0x171a13f (0x9f1f bytes) 2024-05-08T03:19:42.923 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:42.917+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:42.924 INFO:teuthology.orchestra.run.smithi177.stdout:1175 write 0x3805e3b thru 0x3811ded (0xbfb3 bytes) 2024-05-08T03:19:42.925 INFO:teuthology.orchestra.run.smithi177.stdout:1176 punch from 0xf13ac to 0xfbd85, (0xa9d9 bytes) 2024-05-08T03:19:42.925 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.925 INFO:teuthology.orchestra.run.smithi177.stdout:1180 write 0x90ae591 thru 0x90bb035 (0xcaa5 bytes) 2024-05-08T03:19:42.934 INFO:teuthology.orchestra.run.smithi177.stdout:1182 writesame 0x7e2a6b thru 0x7e4a03 data_size 0x1(0x1f99 bytes) 2024-05-08T03:19:42.935 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.935 INFO:teuthology.orchestra.run.smithi177.stdout:1184 read 0xa136a23 thru 0xa1406c8 (0x9ca6 bytes) 2024-05-08T03:19:42.935 INFO:teuthology.orchestra.run.smithi177.stdout:1187 writesame 0x4a1512a thru 0x4a1dcb9 data_size 0x10(0x8b90 bytes) 2024-05-08T03:19:42.939 INFO:teuthology.orchestra.run.smithi177.stdout:1188 write 0x5f8320a thru 0x5f84bac (0x19a3 bytes) 2024-05-08T03:19:42.944 INFO:teuthology.orchestra.run.smithi177.stdout:1189 write 0xa71f2e5 thru 0xa72ce83 (0xdb9f bytes) 2024-05-08T03:19:42.949 INFO:teuthology.orchestra.run.smithi177.stdout:1195 punch from 0x809c5d8 to 0x80a6e1d, (0xa845 bytes) 2024-05-08T03:19:42.950 INFO:teuthology.orchestra.run.smithi177.stdout:1196 punch from 0xa68eea1 to 0xa69a3d5, (0xb534 bytes) 2024-05-08T03:19:42.950 INFO:teuthology.orchestra.run.smithi177.stdout:1197 punch from 0x30320f to 0x30cfb3, (0x9da4 bytes) 2024-05-08T03:19:42.950 INFO:teuthology.orchestra.run.smithi177.stdout:1198 write 0x74ca5f8 thru 0x74d125b (0x6c64 bytes) 2024-05-08T03:19:42.964 INFO:teuthology.orchestra.run.smithi177.stdout:1199 write 0xea6261e thru 0xea62fe5 (0x9c8 bytes) 2024-05-08T03:19:42.967 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:42.967 INFO:teuthology.orchestra.run.smithi177.stdout:1201 read 0x13f31f5 thru 0x13fdbe7 (0xa9f3 bytes) 2024-05-08T03:19:42.974 INFO:teuthology.orchestra.run.smithi177.stdout:1202 trunc from 0xea62fe6 to 0x9a0e2b9 2024-05-08T03:19:43.001 INFO:teuthology.orchestra.run.smithi177.stdout:1203 write 0xb4b9887 thru 0xb4c07db (0x6f55 bytes) 2024-05-08T03:19:43.007 INFO:teuthology.orchestra.run.smithi177.stdout:1204 writesame 0x26aa129 thru 0x26b109d data_size 0x1(0x6f75 bytes) 2024-05-08T03:19:43.015 INFO:teuthology.orchestra.run.smithi177.stdout:1206 write 0x18ec317 thru 0x18f30b0 (0x6d9a bytes) 2024-05-08T03:19:43.019 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:43.018+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:43.020 INFO:teuthology.orchestra.run.smithi177.stdout:1207 read 0x125244b thru 0x125e2f2 (0xbea8 bytes) 2024-05-08T03:19:43.021 INFO:teuthology.orchestra.run.smithi177.stdout:1209 write 0xae2666d thru 0xae2908c (0x2a20 bytes) 2024-05-08T03:19:43.026 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:43.026 INFO:teuthology.orchestra.run.smithi177.stdout:1211 writesame 0x3bf3e64 thru 0x3bfea39 data_size 0x6a(0xabd6 bytes) 2024-05-08T03:19:43.031 INFO:teuthology.orchestra.run.smithi177.stdout:1212 punch from 0xa8ad302 to 0xa8bcc1d, (0xf91b bytes) 2024-05-08T03:19:43.031 INFO:teuthology.orchestra.run.smithi177.stdout:1213 punch from 0xb83f07 to 0xb93286, (0xf37f bytes) 2024-05-08T03:19:43.031 INFO:teuthology.orchestra.run.smithi177.stdout:1214 read 0x4e40b77 thru 0x4e42aae (0x1f38 bytes) 2024-05-08T03:19:43.031 INFO:teuthology.orchestra.run.smithi177.stdout:1217 punch from 0xaab3eee to 0xaab9336, (0x5448 bytes) 2024-05-08T03:19:43.031 INFO:teuthology.orchestra.run.smithi177.stdout:1219 write 0x7d35199 thru 0x7d3fb7c (0xa9e4 bytes) 2024-05-08T03:19:43.039 INFO:teuthology.orchestra.run.smithi177.stdout:1220 write 0x1458f73 thru 0x1466fbc (0xe04a bytes) 2024-05-08T03:19:43.049 INFO:teuthology.orchestra.run.smithi177.stdout:1221 write 0xd3241f4 thru 0xd329494 (0x52a1 bytes) 2024-05-08T03:19:43.056 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.056 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.058 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:43.058 INFO:teuthology.orchestra.run.smithi177.stdout:1223 punch from 0xd328000 to 0xd329495, (0x1495 bytes) 2024-05-08T03:19:43.058 INFO:teuthology.orchestra.run.smithi177.stdout:1224 write 0xb1f8acf thru 0xb1f9fc8 (0x14fa bytes) 2024-05-08T03:19:43.063 INFO:teuthology.orchestra.run.smithi177.stdout:1225 clone 12 order 24 su 4194304 sc 3 2024-05-08T03:19:43.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:43.120 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:43.119+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:43.174 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.174 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:43.223 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:43.221+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:43.256 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:19:43.255+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e39 definitely_dead 0 2024-05-08T03:19:43.323 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:43.321+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:43.327 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.327 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.335 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:43.424 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:43.423+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:43.479 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.480 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:43.532 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:43.530+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:43.593 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.593 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:43.625 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:43.625+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:43.727 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:43.725+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:43.746 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.746 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.755 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:43.839 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:43.837+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:43.901 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:43.901 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:43.901 INFO:teuthology.orchestra.run.smithi177.stderr:marked out osd.7. 2024-05-08T03:19:43.909 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:19:43.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:43.930 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:43.927+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:44.030 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:44.028+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:44.059 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:44.059 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:44.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:44.131 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:44.130+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:44.213 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:44.213 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:44.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:44.265 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:44.263+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:44.326 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:44.326 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:44.333 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:44.332+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:44.334 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:44.434 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:44.433+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:44.481 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:44.481 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:44.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:44.535 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:44.535+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:44.634 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:44.636 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:44.635+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:44.643 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:44.650 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:44.738 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:44.736+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:44.793 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:44.802 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:44.802 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:44.803 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:44.827 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:44.827 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:44.838 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:44.837+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:44.849 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:44.849 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:44.870 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:44.870 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:44.892 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:44.892 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:44.915 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:44.915 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:44.936 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:44.936 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:44.936 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:44.936 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:44.936 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:44.936 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:44.939 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:44.938+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:44.948 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:44.960 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:44.961 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:45.040 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:45.039+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:45.067 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:45.067 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:45.068 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:45.068 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:45.068 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:45.076 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:45.141 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:45.140+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:45.187 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:45.187 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:45.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.981258186000002, 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.188 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:19:45.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:19:45.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:19:45.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.981248736000001, 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:45.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:19:45.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:19:45.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.981235999999999, 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:19:45.196 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.200 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.981225424000002, 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.204 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.981216036999999, 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.205 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:19:45.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:19:45.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.981206857, 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750381822000001, 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750349584999999, 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.217 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:45.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:19:45.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:19:45.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750336255000001, 2024-05-08T03:19:45.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750321281000002, 2024-05-08T03:19:45.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:45.229 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750309972, 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:45.233 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750292302999998, 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:19:45.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:19:45.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:19:45.237 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750277104999999, 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.238 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.750263011000001, 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:19:45.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.9753090380000007, 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.246 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:45.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0524152359999999, 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:19:45.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:19:45.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:19:45.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0524017479999994, 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:19:45.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:45.260 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0523727669999996, 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:19:45.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0523461609999991, 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.265 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6195979000000005, 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:19:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.274 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.275 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.275 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:45.276 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:45.275+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:45.276 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:45.377 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:45.376+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:45.381 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:45.381 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:45.381 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:45.382 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:45.390 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:45.478 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:45.477+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:45.534 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:45.535 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:45.535 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:45.535 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:45.535 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:45.543 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:45.579 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:45.578+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:45.680 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:45.679+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:45.690 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:115 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'65)", 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.604368424, 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014688414, 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083946+0000", 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083947+0000", 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1900000000000001e-07 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.083985+0000", 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7769999999999999e-05 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084001+0000", 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.592e-05 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084009+0000", 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9109999999999993e-06 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084686+0000", 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00067690100000000004 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084703+0000", 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.755e-05 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098569+0000", 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013865446999999999 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098635+0000", 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6095999999999999e-05 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:116 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'65)", 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.604175555000001, 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014525575000000001, 2024-05-08T03:19:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084139+0000", 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084140+0000", 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1900000000000003e-07 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084144+0000", 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6149999999999999e-06 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084153+0000", 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6459999999999999e-06 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084159+0000", 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0079999999999999e-06 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085434+0000", 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001275263 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085455+0000", 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0893000000000001e-05 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098638+0000", 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013182523 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098665+0000", 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7107999999999999e-05 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4206.0:117 2.0 e17/16 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'65)", 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.603507508, 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013907645, 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084807+0000", 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084808+0000", 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-07 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084813+0000", 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2259999999999996e-06 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084822+0000", 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5830000000000007e-06 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.084830+0000", 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9650000000000008e-06 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085897+0000", 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001067496 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.085917+0000", 2024-05-08T03:19:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0080000000000001e-05 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098667+0000", 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012749462 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:14.098715+0000", 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8285999999999998e-05 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:824 3.1a e25/19 3:5e68eb20:::rbd_object_map.11821677beed:head v 25'15, mlcod=25'6)", 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:27.875362+0000", 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.812952618000001, 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092165530000000006, 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875362+0000", 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875362+0000", 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875363+0000", 2024-05-08T03:19:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875370+0000", 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0849999999999999e-06 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875378+0000", 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7149999999999998e-06 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875383+0000", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0420000000000002e-06 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875425+0000", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2568999999999998e-05 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.875437+0000", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1902000000000001e-05 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.884540+0000", 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0091033910000000006 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:27.884579+0000", 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8408000000000002e-05 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2079 3.19 3:99a2a0a7:::rbd_data.1182c698972d.0000000000000310:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.500202+0000", 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.188113187999999, 2024-05-08T03:19:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089751959999999995, 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2079 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500202+0000", 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500202+0000", 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500203+0000", 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3349999999999999e-06 2024-05-08T03:19:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500210+0000", 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6950000000000003e-06 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500223+0000", 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3388e-05 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500229+0000", 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7150000000000003e-06 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500279+0000", 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9855000000000003e-05 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.500358+0000", 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9396999999999999e-05 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.501764+0000", 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0014062499999999999 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509058+0000", 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0072933130000000001 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509075+0000", 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7394e-05 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509129+0000", 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3653999999999998e-05 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.509177+0000", 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8199999999999999e-05 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.714 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11618 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.693988+0000", 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9943267169999999, 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092573900000000008, 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11618 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693988+0000", 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693988+0000", 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693989+0000", 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5699999999999998e-07 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.693990+0000", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.734e-06 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694003+0000", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3064999999999999e-05 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694008+0000", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.843e-06 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.702853+0000", 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0088450109999999998 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.702912+0000", 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8352e-05 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703246+0000", 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033402799999999999 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.718 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11619 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9943004000000002, 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097089580000000002, 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11619 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.719 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694015+0000", 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.84e-07 2024-05-08T03:19:45.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694017+0000", 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8160000000000001e-06 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694023+0000", 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694030+0000", 2024-05-08T03:19:45.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5320000000000003e-06 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703325+0000", 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092952739999999992 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703406+0000", 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1277000000000001e-05 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703723+0000", 2024-05-08T03:19:45.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031717600000000001 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11620 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9942689270000002, 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010112856, 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.723 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11620 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694046+0000", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.51e-07 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694049+0000", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4660000000000002e-06 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694058+0000", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3300000000000005e-06 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694063+0000", 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8690000000000003e-06 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703752+0000", 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0096887310000000008 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.703799+0000", 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7342999999999997e-05 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704159+0000", 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00035966600000000002 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11621 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694069+0000", 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9942456180000008, 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010542635999999999, 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11621 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694069+0000", 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694069+0000", 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694070+0000", 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5100000000000001e-07 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694072+0000", 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8950000000000001e-06 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694083+0000", 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.152e-05 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694110+0000", 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7044000000000001e-05 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704203+0000", 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010093365999999999 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704282+0000", 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8053000000000006e-05 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704612+0000", 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033040700000000001 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11622 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9941783019999999, 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010835354, 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.731 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11622 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:45.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694137+0000", 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5899999999999996e-07 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694140+0000", 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9629999999999998e-06 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694148+0000", 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1599999999999998e-06 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694156+0000", 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0050000000000004e-06 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704651+0000", 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010494852000000001 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704697+0000", 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6140999999999998e-05 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.704972+0000", 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027457399999999998 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11623 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9941493520000009, 2024-05-08T03:19:45.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011222263, 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11623 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694166+0000", 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3300000000000003e-07 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694168+0000", 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4480000000000001e-06 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694189+0000", 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0080000000000001e-05 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694194+0000", 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6790000000000002e-06 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705009+0000", 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010814361999999999 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705073+0000", 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4463999999999998e-05 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705388+0000", 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031479700000000002 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11624 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9941131139999992, 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011578634000000001, 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11624 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694202+0000", 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3499999999999996e-07 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694206+0000", 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9689999999999996e-06 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694212+0000", 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1770000000000003e-06 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694217+0000", 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.87e-06 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705425+0000", 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011207590999999999 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705478+0000", 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2787000000000001e-05 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705780+0000", 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030270500000000001 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11625 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9940913939999998, 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011994640000000001, 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11625 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694224+0000", 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3300000000000003e-07 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694226+0000", 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2859999999999998e-06 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694232+0000", 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8610000000000003e-06 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694237+0000", 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4120000000000003e-06 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705824+0000", 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011587123 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.705900+0000", 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5887e-05 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706218+0000", 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031863799999999999 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.747 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11626 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9940723830000007, 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012453031, 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11626 2024-05-08T03:19:45.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694243+0000", 2024-05-08T03:19:45.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.08e-07 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694245+0000", 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.142e-06 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694251+0000", 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.852e-06 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694256+0000", 2024-05-08T03:19:45.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6360000000000002e-06 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706248+0000", 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011992829999999999 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706336+0000", 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8077000000000004e-05 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706696+0000", 2024-05-08T03:19:45.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000359086 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11627 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9940539929999996, 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01280324, 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11627 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694261+0000", 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999999e-07 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694263+0000", 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9369999999999998e-06 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694269+0000", 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6280000000000002e-06 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694277+0000", 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1799999999999996e-06 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706742+0000", 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012465285 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.706809+0000", 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6917999999999999e-05 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707064+0000", 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025493199999999998 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11628 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:8 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9940139779999999, 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013149357, 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11628 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694301+0000", 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4499999999999997e-07 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694304+0000", 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7379999999999999e-06 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694310+0000", 2024-05-08T03:19:45.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2519999999999996e-06 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694315+0000", 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6009999999999997e-06 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707093+0000", 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012777726 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707151+0000", 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8097999999999998e-05 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.707450+0000", 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00029939700000000002 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12074 3.4 3:216c60a0:::rbd_data.11824ee2abc9.000000000000000a:head [sparse-read 0~5505024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.013854+0000", 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.6744611319999994, 2024-05-08T03:19:45.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010655777999999999, 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12074 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013854+0000", 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013854+0000", 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013857+0000", 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0730000000000001e-06 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013866+0000", 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6840000000000002e-06 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.013988+0000", 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012273300000000001 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014001+0000", 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2505e-05 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014083+0000", 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2636e-05 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014170+0000", 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6953999999999995e-05 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.024510+0000", 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010339193 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12075 3.4 3:2739b017:::rbd_data.11824ee2abc9.000000000000000b:head [sparse-read 0~5505024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.6742863280000009, 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017019135000000001, 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12075 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.765 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014029+0000", 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0200000000000002e-07 2024-05-08T03:19:45.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014033+0000", 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-06 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014045+0000", 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1929e-05 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.014052+0000", 2024-05-08T03:19:45.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5969999999999997e-06 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.024570+0000", 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010517888 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.024681+0000", 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011136799999999999 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.031048+0000", 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.006366521 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13005 3.0 3:01f49529:::rbd_data.118231ecabb9.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.4830095740000004, 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.021952854000000001, 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13005 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205306+0000", 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0999999999999997e-07 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205310+0000", 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8249999999999998e-06 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205319+0000", 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6500000000000002e-06 2024-05-08T03:19:45.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205325+0000", 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3060000000000003e-06 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205368+0000", 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2809000000000003e-05 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205418+0000", 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9920999999999999e-05 2024-05-08T03:19:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.209135+0000", 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037170160000000001 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227134+0000", 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017998962 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227171+0000", 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7462999999999997e-05 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227209+0000", 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8275999999999997e-05 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227258+0000", 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8816000000000002e-05 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.774 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13450 3.0 3:06f1dff9:::rbd_data.118231ecabb9.0000000000000068:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.8473882870000002, 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015476551, 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13450 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:45.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840927+0000", 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7899999999999997e-07 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840932+0000", 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.95e-06 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840943+0000", 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840949+0000", 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5589999999999996e-06 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:45.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.840998+0000", 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9718000000000001e-05 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841058+0000", 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0000999999999997e-05 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.843397+0000", 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0023384510000000001 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856224+0000", 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012827668 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856283+0000", 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8263000000000001e-05 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856350+0000", 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7577999999999996e-05 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:45.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.856403+0000", 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.295e-05 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:45.780 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:45.781 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:45.782 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:45.779+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:45.882 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:45.880+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:45.891 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:45.891 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:45.891 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:45.892 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:45.901 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:45.931 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:19:45.989 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:45.989+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:46.010 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:46.011 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:46.011 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:46.011 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:46.011 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:46.020 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:46.104 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:46.103+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:46.168 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12442 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.9617695939999997, 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036923198999999997, 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12442 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204568+0000", 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6500000000000001e-06 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207214+0000", 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0026454019999999998 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207240+0000", 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6523e-05 2024-05-08T03:19:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207254+0000", 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.43e-05 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207335+0000", 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0395000000000005e-05 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207518+0000", 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182819 2024-05-08T03:19:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208440+0000", 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00092235600000000004 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225462+0000", 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017022065999999999 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241319+0000", 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015856677 2024-05-08T03:19:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241446+0000", 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012764100000000001 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241490+0000", 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.337e-05 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12820 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127022+0000", 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0393139250000001, 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0077392310000000001, 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12820 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127022+0000", 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127022+0000", 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127023+0000", 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.27e-07 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127025+0000", 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.317e-06 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127031+0000", 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4139999999999999e-06 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127036+0000", 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.493e-06 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134343+0000", 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.007306575 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134428+0000", 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5909999999999996e-05 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134762+0000", 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033309499999999999 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12821 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2490368~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0392937010000001, 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083533040000000006, 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12821 2024-05-08T03:19:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9999999999999998e-07 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127048+0000", 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.339e-06 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127055+0000", 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7070000000000003e-06 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127059+0000", 2024-05-08T03:19:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2660000000000003e-06 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134806+0000", 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0077463200000000001 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134926+0000", 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000120184 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135396+0000", 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047008800000000002 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12822 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0392703910000005, 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089581330000000001, 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12822 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5499999999999998e-07 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127069+0000", 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2740000000000002e-06 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127084+0000", 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5495999999999999e-05 2024-05-08T03:19:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127093+0000", 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2779999999999999e-06 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135432+0000", 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083386629999999996 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135511+0000", 2024-05-08T03:19:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9437000000000005e-05 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136024+0000", 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051252999999999997 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12835 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 524288~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0388520020000005, 2024-05-08T03:19:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092348480000000004, 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12835 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127485+0000", 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0500000000000002e-07 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127490+0000", 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1669999999999998e-06 2024-05-08T03:19:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127500+0000", 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0257e-05 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127507+0000", 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4760000000000003e-06 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136070+0000", 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.008563279 2024-05-08T03:19:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136160+0000", 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9445999999999997e-05 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136719+0000", 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000559418 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12836 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2621440~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0388007239999997, 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097351599999999996, 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12836 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127536+0000", 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2399999999999998e-07 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127539+0000", 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1269999999999999e-06 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127547+0000", 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4529999999999996e-06 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127552+0000", 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136745+0000", 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0091925099999999992 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136801+0000", 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6663999999999998e-05 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137271+0000", 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000469412 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12851 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 655360~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0387756350000004, 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010145049999999999, 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12851 2024-05-08T03:19:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6199999999999998e-07 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127563+0000", 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4140000000000001e-06 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127569+0000", 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127574+0000", 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137312+0000", 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097380450000000007 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137391+0000", 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9394000000000006e-05 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137706+0000", 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031446099999999998 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12852 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2752512~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0387566939999999, 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010659394000000001, 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12852 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9000000000000002e-07 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127582+0000", 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127589+0000", 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5919999999999997e-06 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127592+0000", 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8360000000000002e-06 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137740+0000", 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010148121 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137820+0000", 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9524999999999995e-05 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138239+0000", 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00041890800000000001 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12867 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0387390930000002, 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011192809999999999, 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12867 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127599+0000", 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.894e-06 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127605+0000", 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127610+0000", 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8080000000000004e-06 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138262+0000", 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010652069 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138333+0000", 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0634000000000002e-05 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138790+0000", 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00045714900000000001 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12868 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2883584~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0387199949999992, 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011717782, 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12868 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127617+0000", 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.53e-07 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127619+0000", 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4870000000000001e-06 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127625+0000", 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3029999999999996e-06 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127631+0000", 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9900000000000002e-06 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138833+0000", 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011201871 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138902+0000", 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9194000000000002e-05 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139334+0000", 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00043158399999999999 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12883 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0386984249999998, 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012139397, 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12883 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127641+0000", 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127648+0000", 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7989999999999996e-06 2024-05-08T03:19:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127653+0000", 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139365+0000", 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01171205 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139435+0000", 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0135000000000005e-05 2024-05-08T03:19:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139777+0000", 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034227200000000002 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12884 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3014656~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0386770559999992, 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012668127, 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12884 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127660+0000", 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127661+0000", 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.962e-06 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127668+0000", 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.105e-06 2024-05-08T03:19:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127673+0000", 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1649999999999997e-06 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139800+0000", 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012127712000000001 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139884+0000", 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3873000000000001e-05 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140327+0000", 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000442939 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12899 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1048576~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0376438570000008, 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012150764999999999, 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12899 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:19:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128694+0000", 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1739999999999999e-06 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128699+0000", 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1540000000000001e-06 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128714+0000", 2024-05-08T03:19:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5755000000000001e-05 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128722+0000", 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9789999999999993e-06 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140360+0000", 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011637522000000001 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140439+0000", 2024-05-08T03:19:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9297999999999999e-05 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140843+0000", 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00040388300000000001 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12900 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3145728~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0375996230000002, 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012610682999999999, 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12900 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.44e-07 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128740+0000", 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2949999999999998e-06 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128748+0000", 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.43e-06 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128753+0000", 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3399999999999997e-06 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140875+0000", 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012122198000000001 2024-05-08T03:19:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140951+0000", 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6036000000000002e-05 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141347+0000", 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039584 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12915 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1179648~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.037575554, 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013175711999999999, 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12915 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7100000000000002e-07 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128765+0000", 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5300000000000001e-06 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128772+0000", 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7589999999999997e-06 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128778+0000", 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1000000000000003e-06 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141378+0000", 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012600668000000001 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141432+0000", 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3448000000000001e-05 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141936+0000", 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00050473599999999999 2024-05-08T03:19:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12916 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0375505440000001, 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013781595000000001, 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12916 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5900000000000002e-07 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128788+0000", 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2010000000000002e-06 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128795+0000", 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6730000000000003e-06 2024-05-08T03:19:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128800+0000", 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3550000000000004e-06 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141966+0000", 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013165469000000001 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142011+0000", 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5331999999999999e-05 2024-05-08T03:19:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142567+0000", 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000556106 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12931 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1310720~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0375289260000002, 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014420924, 2024-05-08T03:19:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12931 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128808+0000", 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.51e-07 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128810+0000", 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.238e-06 2024-05-08T03:19:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128816+0000", 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128821+0000", 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2499999999999997e-06 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142597+0000", 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013775832999999999 2024-05-08T03:19:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142679+0000", 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1757999999999999e-05 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143228+0000", 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054913200000000003 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12932 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0375083509999996, 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015052658, 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12932 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9900000000000001e-07 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128830+0000", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0999999999999998e-06 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128837+0000", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3080000000000004e-06 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128842+0000", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8910000000000003e-06 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143270+0000", 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014428778999999999 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143335+0000", 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5162999999999999e-05 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143881+0000", 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054501799999999998 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12947 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1441792~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0374882830000001, 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015679150999999999, 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12947 2024-05-08T03:19:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128850+0000", 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.013e-06 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128856+0000", 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0630000000000002e-06 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128861+0000", 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9540000000000003e-06 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143935+0000", 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015074086 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144017+0000", 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1118000000000005e-05 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144527+0000", 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051051099999999995 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12948 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3538944~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.0374680620000003, 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016271184000000001, 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12948 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128869+0000", 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128871+0000", 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128877+0000", 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:19:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128882+0000", 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2719999999999997e-06 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144554+0000", 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015671378999999999 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144626+0000", 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2265e-05 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145139+0000", 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051343000000000005 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.258 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:46.260 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:46.260 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:46.259+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:46.360 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:46.358+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:46.367 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-05-08T03:19:46.414 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:46.415 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:46.415 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:46.415 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:46.431 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:46.461 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:46.460+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:46.549 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:46.549 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:46.550 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:46.550 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:46.550 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:46.560 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:46.561 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:46.560+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:46.647 INFO:teuthology.orchestra.run.smithi177.stdout:1226 read 0xc8c6479 thru 0xc8cf381 (0x8f09 bytes) 2024-05-08T03:19:46.647 INFO:teuthology.orchestra.run.smithi177.stdout:1227 punch from 0x71172e5 to 0x7126ef5, (0xfc10 bytes) 2024-05-08T03:19:46.651 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:46.651 INFO:teuthology.orchestra.run.smithi177.stdout:1231 punch from 0x69b6f33 to 0x69bc763, (0x5830 bytes) 2024-05-08T03:19:46.651 INFO:teuthology.orchestra.run.smithi177.stdout:1232 clone 13 order 23 su 131072 sc 9 2024-05-08T03:19:46.662 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:46.661+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1495 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2600960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.510245510000001, 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015832629000000001, 2024-05-08T03:19:46.671 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1495 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158482+0000", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.17e-07 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158483+0000", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9599999999999998e-07 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158486+0000", 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9450000000000002e-06 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158488+0000", 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174237+0000", 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015748306 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174275+0000", 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8578999999999998e-05 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174315+0000", 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9495e-05 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1496 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2609152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:46.675 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.510236600999999, 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015926348999999999, 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1496 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158492+0000", 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3600000000000002e-07 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158495+0000", 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8339999999999999e-06 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158497+0000", 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2230000000000001e-06 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174338+0000", 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015840481 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174376+0000", 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8642e-05 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174417+0000", 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1025000000000001e-05 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1497 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.510224711999999, 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016000936, 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1497 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3999999999999998e-07 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158504+0000", 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.071e-06 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158508+0000", 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3440000000000001e-06 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158513+0000", 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7899999999999996e-06 2024-05-08T03:19:46.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174433+0000", 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015919098999999999 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174468+0000", 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5447999999999997e-05 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174504+0000", 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5944e-05 2024-05-08T03:19:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11712 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973904709999999, 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016206332, 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.684 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11712 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694826+0000", 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1440000000000001e-06 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694833+0000", 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8839999999999994e-06 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694839+0000", 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.395e-06 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.710617+0000", 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015777899000000001 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.710705+0000", 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8290999999999999e-05 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711029+0000", 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00032433599999999999 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11713 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973882573999999, 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016660854999999999, 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.688 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11713 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:46.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.84e-07 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694847+0000", 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.46e-06 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694853+0000", 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5549999999999999e-06 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694858+0000", 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8409999999999999e-06 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711071+0000", 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016213165000000002 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711160+0000", 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8337999999999993e-05 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711506+0000", 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034621199999999998 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11714 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973863977000001, 2024-05-08T03:19:46.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017110975, 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11714 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.53e-07 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694866+0000", 2024-05-08T03:19:46.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7749999999999999e-06 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694869+0000", 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2839999999999998e-06 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694872+0000", 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.559e-06 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711545+0000", 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016672900000000001 2024-05-08T03:19:46.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711626+0000", 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1076999999999996e-05 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711975+0000", 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034912700000000002 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11715 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 884736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973852981, 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017762351999999999, 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11715 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694876+0000", 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694881+0000", 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5650000000000003e-06 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694885+0000", 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2039999999999999e-06 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712010+0000", 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017124784000000001 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712104+0000", 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4618999999999997e-05 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712637+0000", 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00053277399999999996 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.700 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11716 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.9738367, 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018326299000000001, 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11716 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2000000000000001e-07 2024-05-08T03:19:46.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694893+0000", 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694899+0000", 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6890000000000001e-06 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694903+0000", 2024-05-08T03:19:46.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3510000000000004e-06 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712677+0000", 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017773487000000001 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712790+0000", 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011356100000000001 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713217+0000", 2024-05-08T03:19:46.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042686900000000002 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11717 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973818419000001, 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018801206000000001, 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.705 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11717 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694910+0000", 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694912+0000", 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.001e-06 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694917+0000", 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2510000000000002e-06 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694922+0000", 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6009999999999997e-06 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713270+0000", 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018348738999999999 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713346+0000", 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5591000000000002e-05 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713711+0000", 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00036462499999999999 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11718 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973800616, 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019351284, 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11718 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4799999999999999e-07 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694929+0000", 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8449999999999999e-06 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694935+0000", 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5470000000000004e-06 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694940+0000", 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8099999999999997e-06 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713760+0000", 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018820130000000001 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713881+0000", 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012139800000000001 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714278+0000", 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039720599999999999 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11719 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973782701999999, 2024-05-08T03:19:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019846827000000001, 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11719 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6399999999999998e-07 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694947+0000", 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8369999999999999e-06 2024-05-08T03:19:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694960+0000", 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2758000000000001e-05 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694965+0000", 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7210000000000002e-06 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714344+0000", 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019379181999999998 2024-05-08T03:19:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714442+0000", 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8158000000000004e-05 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714792+0000", 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034980699999999999 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11720 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973755868, 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022940031999999999, 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11720 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3799999999999998e-07 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694975+0000", 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2979999999999999e-06 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694981+0000", 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1399999999999997e-06 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694986+0000", 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.665e-06 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714835+0000", 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019848141999999999 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714919+0000", 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4155999999999999e-05 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.717912+0000", 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0029931929999999999 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11721 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973722589999999, 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025391021999999999, 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11721 2024-05-08T03:19:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0499999999999999e-07 2024-05-08T03:19:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695007+0000", 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.761e-06 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695015+0000", 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4529999999999996e-06 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695018+0000", 2024-05-08T03:19:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7910000000000002e-06 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.717956+0000", 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022937760000000001 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.718014+0000", 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7535999999999999e-05 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720396+0000", 2024-05-08T03:19:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.002382416 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11722 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973704752, 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028041909, 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11722 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.48e-07 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695025+0000", 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3820000000000001e-06 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695028+0000", 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.241e-06 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695030+0000", 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4210000000000002e-06 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720442+0000", 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025412039000000001 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720499+0000", 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6721000000000001e-05 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723065+0000", 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0025658569999999999 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11723 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973694445, 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028402541, 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11723 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695034+0000", 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9400000000000001e-07 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695046+0000", 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2328e-05 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695057+0000", 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0789e-05 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695062+0000", 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2909999999999998e-06 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723110+0000", 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028047704999999999 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723161+0000", 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1598000000000003e-05 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723436+0000", 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027453599999999997 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11724 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.973651046000001, 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028762263999999999, 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11724 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9499999999999998e-07 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695079+0000", 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5409999999999999e-06 2024-05-08T03:19:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695092+0000", 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3203999999999999e-05 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695096+0000", 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2209999999999999e-06 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723482+0000", 2024-05-08T03:19:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028385778 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723533+0000", 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0966e-05 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723839+0000", 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030625900000000002 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:12446 3.c e33/19 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:head v 33'23, mlcod=33'21)", 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.444038288, 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016492471000000002, 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224691+0000", 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8169999999999999e-06 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229419+0000", 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0047272959999999998 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229434+0000", 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5432000000000001e-05 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229458+0000", 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3796000000000001e-05 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229542+0000", 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4031999999999998e-05 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229559+0000", 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6820000000000002e-05 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241051+0000", 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011492005 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241182+0000", 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000131273 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12953 3.18 3:18c50615:::rbd_data.1182ca0d3624.0000000000000002:a [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5395335540000001, 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016630718999999999, 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12953 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129195+0000", 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0900000000000002e-07 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129197+0000", 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3089999999999998e-06 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129205+0000", 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6389999999999992e-06 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129211+0000", 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3550000000000001e-06 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145049+0000", 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015838181 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145160+0000", 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011112600000000001 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145825+0000", 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00066469999999999995 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12954 3.18 3:18c50615:::rbd_data.1182ca0d3624.0000000000000002:a [sparse-read 4063232~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.5395100530000008, 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017202562000000001, 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12954 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129224+0000", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8640000000000001e-06 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129232+0000", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5259999999999996e-06 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129237+0000", 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145863+0000", 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016625859999999999 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145960+0000", 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7036999999999999e-05 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146420+0000", 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00046035500000000002 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13451 3.7 3:e3f1c4ac:::rbd_data.118231ecabb9.0000000000000069:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.8273797419999998, 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022473401, 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13451 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841350+0000", 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.545e-06 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841355+0000", 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8420000000000001e-06 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841367+0000", 2024-05-08T03:19:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1600000000000001e-05 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841373+0000", 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4309999999999999e-06 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841433+0000", 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0028000000000002e-05 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841515+0000", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1508000000000006e-05 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.845379+0000", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038636790000000001 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.862654+0000", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017274982000000001 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863725+0000", 2024-05-08T03:19:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010711029999999999 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863777+0000", 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2753000000000001e-05 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863821+0000", 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3930000000000001e-05 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:46.756 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:46.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:46.771 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:19:46.803 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:46.801+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:46.864 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:46.862+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:46.866 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:46.866 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:46.866 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:46.866 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:46.875 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:46.965 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:46.964+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:46.973 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:47.021 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:47.021 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:47.021 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:47.021 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:47.021 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:47.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:47.076 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:47.074+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:47.095 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:47.095 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:47.112 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:47.113 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:47.129 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:47.129 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.268764068999999, 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:19:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.268421890999999, 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:19:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:19:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:19:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:19:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.268383059000001, 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:19:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.268337026000001, 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.267319833999998, 2024-05-08T03:19:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:19:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:19:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.267283583000001, 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.170 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.267256283999998, 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.174 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:19:47.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.267226591, 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.180 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:47.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.185 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.267201075999999, 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:19:47.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.191 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.257086910000002, 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:19:47.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:19:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:19:47.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.196 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.197 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004748065000001, 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.201 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004729734000001, 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.202 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:19:47.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004705272999999, 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.206 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:19:47.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004678930000001, 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:19:47.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.213 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.214 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004661539000001, 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.215 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.216 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:47.219 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004644257999999, 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:47.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:19:47.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004616820999999, 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.224 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004598862000002, 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.228 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:19:47.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:19:47.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:19:47.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004581775999998, 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:47.240 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.004564792, 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:47.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:19:47.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:47.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.244 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:47.245 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:47.245 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:47.246 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:47.246 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:47.247 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:47.247 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:47.247 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:47.248 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:47.245+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:47.248 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:47.248 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:47.249 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:47.249 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:47.249 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:47.250 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:47.347 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:47.345+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:47.394 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:47.395 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:47.403 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:47.488 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:47.487+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:47.548 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:47.546+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:47.549 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:47.549 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:47.558 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:47.649 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:47.647+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:47.703 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:47.703 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:47.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:47.753 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:47.752+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:47.848 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone12 from 0xd329495 (overlap 0xd329495) to 0x5476ea2 2024-05-08T03:19:47.851 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:47.850+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:47.865 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:47.865 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:47.875 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:47.951 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:47.951+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:47.963 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-05-08T03:19:47.984 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:47.984 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:47.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:48.056 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:48.055+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:48.098 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.098 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.107 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:48.157 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:48.157+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:48.194 INFO:teuthology.orchestra.run.smithi177.stdout:1233 punch from 0x7dcb7ab to 0x7dd8994, (0xd1e9 bytes) 2024-05-08T03:19:48.203 INFO:teuthology.orchestra.run.smithi177.stdout:1234 trunc from 0xd329495 to 0x8d35935 2024-05-08T03:19:48.221 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.221 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.232 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:48.252 INFO:teuthology.orchestra.run.smithi177.stdout:1236 read 0x4ad13ce thru 0x4ad638b (0x4fbe bytes) 2024-05-08T03:19:48.252 INFO:teuthology.orchestra.run.smithi177.stdout:1237 punch from 0x5a38646 to 0x5a407f0, (0x81aa bytes) 2024-05-08T03:19:48.253 INFO:teuthology.orchestra.run.smithi177.stdout:1238 read 0x6ec046d thru 0x6ec8ec5 (0x8a59 bytes) 2024-05-08T03:19:48.253 INFO:teuthology.orchestra.run.smithi177.stdout:1240 write 0x7313d0f thru 0x7321ddb (0xe0cd bytes) 2024-05-08T03:19:48.259 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:48.258+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:48.336 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.336 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.345 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:48.360 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:48.359+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:48.447 INFO:teuthology.orchestra.run.smithi177.stdout:1241 writesame 0x55be21a thru 0x55cabf7 data_size 0x63(0xc9de bytes) 2024-05-08T03:19:48.449 INFO:teuthology.orchestra.run.smithi177.stdout:1242 punch from 0x4d0c379 to 0x4d12bcc, (0x6853 bytes) 2024-05-08T03:19:48.449 INFO:teuthology.orchestra.run.smithi177.stdout:1243 punch from 0x666b1a7 to 0x6679df6, (0xec4f bytes) 2024-05-08T03:19:48.449 INFO:teuthology.orchestra.run.smithi177.stdout:1244 write 0x2602258 thru 0x2608fcd (0x6d76 bytes) 2024-05-08T03:19:48.449 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.450 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.460 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:48.459+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:48.460 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:48.486 INFO:teuthology.orchestra.run.smithi177.stdout:1245 flatten 2024-05-08T03:19:48.561 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:48.560+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:48.606 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.606 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.612 INFO:teuthology.orchestra.run.smithi177.stdout:1246 write 0x843ea50 thru 0x844972d (0xacde bytes) 2024-05-08T03:19:48.614 INFO:teuthology.orchestra.run.smithi177.stdout:1247 write 0x3ea76db thru 0x3eb5136 (0xda5c bytes) 2024-05-08T03:19:48.617 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:48.625 INFO:teuthology.orchestra.run.smithi177.stdout:1249 write 0xa166270 thru 0xa16b670 (0x5401 bytes) 2024-05-08T03:19:48.628 INFO:teuthology.orchestra.run.smithi177.stdout:1250 write 0x4c9a1df thru 0x4ca46c3 (0xa4e5 bytes) 2024-05-08T03:19:48.629 INFO:teuthology.orchestra.run.smithi177.stdout:1251 write 0x32a3288 thru 0x32a536b (0x20e4 bytes) 2024-05-08T03:19:48.639 INFO:teuthology.orchestra.run.smithi177.stdout:1252 trunc from 0xa16b671 to 0x3462e36 2024-05-08T03:19:48.657 INFO:teuthology.orchestra.run.smithi177.stdout:1253 punch from 0x108576e to 0x108ab56, (0x53e8 bytes) 2024-05-08T03:19:48.657 INFO:teuthology.orchestra.run.smithi177.stdout:1254 punch from 0x2b52792 to 0x2b549b0, (0x221e bytes) 2024-05-08T03:19:48.662 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:48.661+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:48.672 INFO:teuthology.orchestra.run.smithi177.stdout:1255 writesame 0x9bddfee thru 0x9bec3d2 data_size 0x1(0xe3e5 bytes) 2024-05-08T03:19:48.685 INFO:teuthology.orchestra.run.smithi177.stdout:1256 write 0xed6f0ce thru 0xed7ba94 (0xc9c7 bytes) 2024-05-08T03:19:48.689 INFO:teuthology.orchestra.run.smithi177.stdout:1257 write 0x45147f4 thru 0x451bd6e (0x757b bytes) 2024-05-08T03:19:48.690 INFO:teuthology.orchestra.run.smithi177.stdout:1258 write 0x300aa51 thru 0x3011837 (0x6de7 bytes) 2024-05-08T03:19:48.692 INFO:teuthology.orchestra.run.smithi177.stdout:1259 write 0xb417bb0 thru 0xb41d319 (0x576a bytes) 2024-05-08T03:19:48.695 INFO:teuthology.orchestra.run.smithi177.stdout:1262 read 0xbb5959e thru 0xbb6070a (0x716d bytes) 2024-05-08T03:19:48.695 INFO:teuthology.orchestra.run.smithi177.stdout:1265 write 0x47a0958 thru 0x47a7e82 (0x752b bytes) 2024-05-08T03:19:48.696 INFO:teuthology.orchestra.run.smithi177.stdout:1266 read 0xa4571f thru 0xa518ea (0xc1cc bytes) 2024-05-08T03:19:48.698 INFO:teuthology.orchestra.run.smithi177.stdout:1268 read 0xd11c0f1 thru 0xd12639b (0xa2ab bytes) 2024-05-08T03:19:48.698 INFO:teuthology.orchestra.run.smithi177.stdout:1269 read 0xc0a97e2 thru 0xc0b04b1 (0x6cd0 bytes) 2024-05-08T03:19:48.698 INFO:teuthology.orchestra.run.smithi177.stdout:1270 write 0x655b94a thru 0x6564616 (0x8ccd bytes) 2024-05-08T03:19:48.701 INFO:teuthology.orchestra.run.smithi177.stdout:1272 write 0x86de552 thru 0x86e28b3 (0x4362 bytes) 2024-05-08T03:19:48.705 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xed4e950 2024-05-08T03:19:48.705 INFO:teuthology.orchestra.run.smithi177.stdout:1273 trunc from 0xed7ba95 to 0xed4e950 2024-05-08T03:19:48.710 INFO:teuthology.orchestra.run.smithi177.stdout:1274 writesame 0x36f1cd7 thru 0x36f321b data_size 0x2d(0x1545 bytes) 2024-05-08T03:19:48.718 INFO:teuthology.orchestra.run.smithi177.stdout:1276 trunc from 0xed4e950 to 0x9a8c79a 2024-05-08T03:19:48.736 INFO:teuthology.orchestra.run.smithi177.stdout:1280 trunc from 0x9a8c79a to 0x59b0db5 2024-05-08T03:19:48.751 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.751 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.755 INFO:teuthology.orchestra.run.smithi177.stdout:1282 writesame 0x7aede94 thru 0x7af4785 data_size 0x65(0x68f2 bytes) 2024-05-08T03:19:48.755 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.755 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.763 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:48.764 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:48.762+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:48.767 INFO:teuthology.orchestra.run.smithi177.stdout:1286 write 0xd83e50d thru 0xd83fd46 (0x183a bytes) 2024-05-08T03:19:48.780 INFO:teuthology.orchestra.run.smithi177.stdout:1287 trunc from 0xd83fd47 to 0x40d9528 2024-05-08T03:19:48.786 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:19:48.806 INFO:teuthology.orchestra.run.smithi177.stdout:1289 read 0xc4ba8f thru 0xc54f38 (0x94aa bytes) 2024-05-08T03:19:48.817 INFO:teuthology.orchestra.run.smithi177.stdout:1291 write 0x6724d1f thru 0x6728001 (0x32e3 bytes) 2024-05-08T03:19:48.821 INFO:teuthology.orchestra.run.smithi177.stdout:1292 read 0x1653ef9 thru 0x165eaae (0xabb6 bytes) 2024-05-08T03:19:48.834 INFO:teuthology.orchestra.run.smithi177.stdout:1294 write 0xb078fcd thru 0xb0811ef (0x8223 bytes) 2024-05-08T03:19:48.838 INFO:teuthology.orchestra.run.smithi177.stdout:1295 write 0x3778ea9 thru 0x377ed73 (0x5ecb bytes) 2024-05-08T03:19:48.840 INFO:teuthology.orchestra.run.smithi177.stdout:1296 read 0x3d7e8af thru 0x3d85394 (0x6ae6 bytes) 2024-05-08T03:19:48.848 INFO:teuthology.orchestra.run.smithi177.stdout:1298 write 0xd56ff3b thru 0xd573a8e (0x3b54 bytes) 2024-05-08T03:19:48.852 INFO:teuthology.orchestra.run.smithi177.stdout:1299 writesame 0x80e32f7 thru 0x80e97e5 data_size 0x57(0x64ef bytes) 2024-05-08T03:19:48.855 INFO:teuthology.orchestra.run.smithi177.stdout:1300 writesame 0x60f796c thru 0x60fb59c data_size 0x1(0x3c31 bytes) 2024-05-08T03:19:48.864 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:48.863+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:48.865 INFO:teuthology.orchestra.run.smithi177.stdout:1301 punch from 0x999279e to 0x999db5c, (0xb3be bytes) 2024-05-08T03:19:48.865 INFO:teuthology.orchestra.run.smithi177.stdout:1302 write 0x908ff8e thru 0x9092e3d (0x2eb0 bytes) 2024-05-08T03:19:48.868 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.871 INFO:teuthology.orchestra.run.smithi177.stdout:1305 trunc from 0xd573a8f to 0xaae5bf6 2024-05-08T03:19:48.883 INFO:teuthology.orchestra.run.smithi177.stdout:1306 trunc from 0xaae5bf6 to 0x62f0f24 2024-05-08T03:19:48.898 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:48.898 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:48.902 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.902 INFO:teuthology.orchestra.run.smithi177.stdout:1308 write 0x3d57470 thru 0x3d5b6ea (0x427b bytes) 2024-05-08T03:19:48.904 INFO:teuthology.orchestra.run.smithi177.stdout:1309 trunc from 0x62f0f24 to 0x5e4378d 2024-05-08T03:19:48.909 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:48.923 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6] out_osds: [4, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:48.923 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:48.924 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 5, 6] 2024-05-08T03:19:48.924 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-05-08T03:19:48.926 INFO:teuthology.orchestra.run.smithi177.stdout:1311 write 0xc06b606 thru 0xc07608c (0xaa87 bytes) 2024-05-08T03:19:48.929 INFO:teuthology.orchestra.run.smithi177.stdout:1312 writesame 0x3b37a82 thru 0x3b3d7c5 data_size 0x1(0x5d44 bytes) 2024-05-08T03:19:48.944 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.950 INFO:teuthology.orchestra.run.smithi177.stdout:1314 trunc from 0xc07608d to 0x7d6085c 2024-05-08T03:19:48.965 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:48.964+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:48.974 INFO:teuthology.orchestra.run.smithi177.stdout:1316 writesame 0x983977e thru 0x9847021 data_size 0xbc(0xd8a4 bytes) 2024-05-08T03:19:48.985 INFO:teuthology.orchestra.run.smithi177.stdout:1317 writesame 0xd7b9e38 thru 0xd7bff3c data_size 0x11(0x6105 bytes) 2024-05-08T03:19:48.985 INFO:teuthology.orchestra.run.smithi177.stdout:1319 write 0x5127051 thru 0x513657d (0xf52d bytes) 2024-05-08T03:19:48.987 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.987 INFO:teuthology.orchestra.run.smithi177.stdout:1324 read 0xb1dcb01 thru 0xb1de9f9 (0x1ef9 bytes) 2024-05-08T03:19:48.987 INFO:teuthology.orchestra.run.smithi177.stdout:1325 write 0x3ad0d02 thru 0x3add2b6 (0xc5b5 bytes) 2024-05-08T03:19:48.991 INFO:teuthology.orchestra.run.smithi177.stdout:1326 punch from 0x97dea69 to 0x97e30a5, (0x463c bytes) 2024-05-08T03:19:48.991 INFO:teuthology.orchestra.run.smithi177.stdout:1327 punch from 0x49f6d95 to 0x49fd276, (0x64e1 bytes) 2024-05-08T03:19:48.991 INFO:teuthology.orchestra.run.smithi177.stdout:1329 write 0xb0d6346 thru 0xb0df46d (0x9128 bytes) 2024-05-08T03:19:48.996 INFO:teuthology.orchestra.run.smithi177.stdout:1331 read 0x874d823 thru 0x8753e31 (0x660f bytes) 2024-05-08T03:19:48.996 INFO:teuthology.orchestra.run.smithi177.stdout:1332 read 0x343fd0f thru 0x3446374 (0x6666 bytes) 2024-05-08T03:19:48.996 INFO:teuthology.orchestra.run.smithi177.stdout:1333 read 0x3914ab6 thru 0x3921e53 (0xd39e bytes) 2024-05-08T03:19:48.997 INFO:teuthology.orchestra.run.smithi177.stdout:1335 write 0x5c6a2e1 thru 0x5c722e6 (0x8006 bytes) 2024-05-08T03:19:48.999 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:48.999 INFO:teuthology.orchestra.run.smithi177.stdout:1339 read 0x4123132 thru 0x412449f (0x136e bytes) 2024-05-08T03:19:49.010 INFO:teuthology.orchestra.run.smithi177.stdout:1340 trunc from 0xd7bff3d to 0x4ce74fa 2024-05-08T03:19:49.036 INFO:teuthology.orchestra.run.smithi177.stdout:1341 writesame 0xa1c54d7 thru 0xa1c7733 data_size 0x1(0x225d bytes) 2024-05-08T03:19:49.036 INFO:teuthology.orchestra.run.smithi177.stdout:1342 write 0x34f4498 thru 0x3503df7 (0xf960 bytes) 2024-05-08T03:19:49.038 INFO:teuthology.orchestra.run.smithi177.stdout:1343 write 0x9ef97a0 thru 0x9efc689 (0x2eea bytes) 2024-05-08T03:19:49.041 INFO:teuthology.orchestra.run.smithi177.stdout:1344 punch from 0x921a84a to 0x9225ac9, (0xb27f bytes) 2024-05-08T03:19:49.053 INFO:teuthology.orchestra.run.smithi177.stdout:1345 writesame 0xe3dd228 thru 0xe3dec8e data_size 0x3(0x1a67 bytes) 2024-05-08T03:19:49.053 INFO:teuthology.orchestra.run.smithi177.stdout:1346 read 0xb041409 thru 0xb041cfa (0x8f2 bytes) 2024-05-08T03:19:49.053 INFO:teuthology.orchestra.run.smithi177.stdout:1347 write 0x3563469 thru 0x3567956 (0x44ee bytes) 2024-05-08T03:19:49.059 INFO:teuthology.orchestra.run.smithi177.stdout:1348 trunc from 0xe3dec8f to 0xaca0a34 2024-05-08T03:19:49.061 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:49.061 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:49.066 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:49.065+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:49.067 INFO:teuthology.orchestra.run.smithi177.stdout:1349 read 0x2e6cda7 thru 0x2e73165 (0x63bf bytes) 2024-05-08T03:19:49.068 INFO:teuthology.orchestra.run.smithi177.stdout:1350 read 0xaacfd26 thru 0xaad8cc9 (0x8fa4 bytes) 2024-05-08T03:19:49.068 INFO:teuthology.orchestra.run.smithi177.stdout:1351 writesame 0xac25d9f thru 0xac27303 data_size 0x1(0x1565 bytes) 2024-05-08T03:19:49.073 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:49.074 INFO:teuthology.orchestra.run.smithi177.stdout:1352 write 0x2a6edfb thru 0x2a70a16 (0x1c1c bytes) 2024-05-08T03:19:49.076 INFO:teuthology.orchestra.run.smithi177.stdout:1353 write 0x93621 thru 0x9dab0 (0xa490 bytes) 2024-05-08T03:19:49.078 INFO:teuthology.orchestra.run.smithi177.stdout:1356 writesame 0x8636099 thru 0x863b4e1 data_size 0x1(0x5449 bytes) 2024-05-08T03:19:49.089 INFO:teuthology.orchestra.run.smithi177.stdout:1357 punch from 0x385566c to 0x385d288, (0x7c1c bytes) 2024-05-08T03:19:49.095 INFO:teuthology.orchestra.run.smithi177.stdout:1358 writesame 0xb07b8e6 thru 0xb07c462 data_size 0x1(0xb7d bytes) 2024-05-08T03:19:49.096 INFO:teuthology.orchestra.run.smithi177.stdout:1360 read 0xac38b61 thru 0xac3f10c (0x65ac bytes) 2024-05-08T03:19:49.096 INFO:teuthology.orchestra.run.smithi177.stdout:1362 write 0x95794b2 thru 0x957acd4 (0x1823 bytes) 2024-05-08T03:19:49.099 INFO:teuthology.orchestra.run.smithi177.stdout:1363 read 0x3d13f7e thru 0x3d1f086 (0xb109 bytes) 2024-05-08T03:19:49.099 INFO:teuthology.orchestra.run.smithi177.stdout:1364 punch from 0x3949fa7 to 0x39541f5, (0xa24e bytes) 2024-05-08T03:19:49.099 INFO:teuthology.orchestra.run.smithi177.stdout:1365 read 0xae191a0 thru 0xae234da (0xa33b bytes) 2024-05-08T03:19:49.100 INFO:teuthology.orchestra.run.smithi177.stdout:1369 punch from 0xa0dbc72 to 0xa0e8f86, (0xd314 bytes) 2024-05-08T03:19:49.100 INFO:teuthology.orchestra.run.smithi177.stdout:1371 write 0x49f4e9a thru 0x49f5495 (0x5fc bytes) 2024-05-08T03:19:49.116 INFO:teuthology.orchestra.run.smithi177.stdout:1372 trunc from 0xb07c463 to 0x112b701 2024-05-08T03:19:49.156 INFO:teuthology.orchestra.run.smithi177.stdout:1374 write 0x39ea55c thru 0x39f7df9 (0xd89e bytes) 2024-05-08T03:19:49.157 INFO:teuthology.orchestra.run.smithi177.stdout:1375 write 0x35fd9b2 thru 0x360193c (0x3f8b bytes) 2024-05-08T03:19:49.162 INFO:teuthology.orchestra.run.smithi177.stdout:1376 punch from 0x1e64ffa to 0x1e6f405, (0xa40b bytes) 2024-05-08T03:19:49.162 INFO:teuthology.orchestra.run.smithi177.stdout:1377 writesame 0x1e4a49e thru 0x1e4c8b6 data_size 0x1(0x2419 bytes) 2024-05-08T03:19:49.168 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:49.167+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:49.177 INFO:teuthology.orchestra.run.smithi177.stdout:1378 writesame 0x806f957 thru 0x80751fb data_size 0x1(0x58a5 bytes) 2024-05-08T03:19:49.190 INFO:teuthology.orchestra.run.smithi177.stdout:1380 write 0xe47409c thru 0xe47896e (0x48d3 bytes) 2024-05-08T03:19:49.193 INFO:teuthology.orchestra.run.smithi177.stdout:1381 read 0x645075 thru 0x6482f9 (0x3285 bytes) 2024-05-08T03:19:49.194 INFO:teuthology.orchestra.run.smithi177.stdout:1384 write 0x5612f8c thru 0x561f49f (0xc514 bytes) 2024-05-08T03:19:49.198 INFO:teuthology.orchestra.run.smithi177.stdout:1386 punch from 0x18eed72 to 0x18f80e7, (0x9375 bytes) 2024-05-08T03:19:49.198 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.198 INFO:teuthology.orchestra.run.smithi177.stdout:1388 read 0xb933048 thru 0xb93d626 (0xa5df bytes) 2024-05-08T03:19:49.198 INFO:teuthology.orchestra.run.smithi177.stdout:1389 read 0xda7912e thru 0xda79d28 (0xbfb bytes) 2024-05-08T03:19:49.198 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.198 INFO:teuthology.orchestra.run.smithi177.stdout:1391 read 0x9319fa8 thru 0x931ac84 (0xcdd bytes) 2024-05-08T03:19:49.199 INFO:teuthology.orchestra.run.smithi177.stdout:1392 write 0x6c477a2 thru 0x6c4863d (0xe9c bytes) 2024-05-08T03:19:49.200 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.200 INFO:teuthology.orchestra.run.smithi177.stdout:1397 read 0x18502d7 thru 0x185ef03 (0xec2d bytes) 2024-05-08T03:19:49.201 INFO:teuthology.orchestra.run.smithi177.stdout:1398 read 0x85e699c thru 0x85e7116 (0x77b bytes) 2024-05-08T03:19:49.201 INFO:teuthology.orchestra.run.smithi177.stdout:1399 writesame 0x3b8e6b1 thru 0x3b90d51 data_size 0xb(0x26a1 bytes) 2024-05-08T03:19:49.203 INFO:teuthology.orchestra.run.smithi177.stdout:1401 read 0xaf8ec85 thru 0xaf97e93 (0x920f bytes) 2024-05-08T03:19:49.203 INFO:teuthology.orchestra.run.smithi177.stdout:1402 read 0xb12c891 thru 0xb13537c (0x8aec bytes) 2024-05-08T03:19:49.203 INFO:teuthology.orchestra.run.smithi177.stdout:1403 write 0x13141e4 thru 0x1323e23 (0xfc40 bytes) 2024-05-08T03:19:49.205 INFO:teuthology.orchestra.run.smithi177.stdout:1404 write 0xd428228 thru 0xd42bd8c (0x3b65 bytes) 2024-05-08T03:19:49.208 INFO:teuthology.orchestra.run.smithi177.stdout:1405 writesame 0xa5f925a thru 0xa6011a9 data_size 0x70(0x7f50 bytes) 2024-05-08T03:19:49.208 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:49.208 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:49.215 INFO:teuthology.orchestra.run.smithi177.stdout:1406 trunc from 0xe47896f to 0xd91599f 2024-05-08T03:19:49.219 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:49.224 INFO:teuthology.orchestra.run.smithi177.stdout:1407 write 0x2053ffe thru 0x20545a8 (0x5ab bytes) 2024-05-08T03:19:49.226 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.226 INFO:teuthology.orchestra.run.smithi177.stdout:1411 punch from 0x12aed83 to 0x12b45b7, (0x5834 bytes) 2024-05-08T03:19:49.226 INFO:teuthology.orchestra.run.smithi177.stdout:1412 trunc from 0xd91599f to 0xd547234 2024-05-08T03:19:49.243 INFO:teuthology.orchestra.run.smithi177.stdout:1413 read 0x9ddc583 thru 0x9ddc657 (0xd5 bytes) 2024-05-08T03:19:49.244 INFO:teuthology.orchestra.run.smithi177.stdout:1414 writesame 0x7bf77d thru 0x7c7d99 data_size 0x13(0x861d bytes) 2024-05-08T03:19:49.246 INFO:teuthology.orchestra.run.smithi177.stdout:1415 read 0xaa200c5 thru 0xaa2bd39 (0xbc75 bytes) 2024-05-08T03:19:49.246 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.246 INFO:teuthology.orchestra.run.smithi177.stdout:1418 write 0x4a748f thru 0x4a9f53 (0x2ac5 bytes) 2024-05-08T03:19:49.249 INFO:teuthology.orchestra.run.smithi177.stdout:1419 trunc from 0xd547234 to 0xc2892c8 2024-05-08T03:19:49.250 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:49.257 INFO:teuthology.orchestra.run.smithi177.stdout:1420 write 0x8dfe5b2 thru 0x8e01e93 (0x38e2 bytes) 2024-05-08T03:19:49.260 INFO:teuthology.orchestra.run.smithi177.stdout:1422 read 0xa43691e thru 0xa4423dc (0xbabf bytes) 2024-05-08T03:19:49.268 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:49.267+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:49.269 INFO:teuthology.orchestra.run.smithi177.stdout:1424 writesame 0xdc43580 thru 0xdc46272 data_size 0x1(0x2cf3 bytes) 2024-05-08T03:19:49.269 INFO:teuthology.orchestra.run.smithi177.stdout:1425 read 0x7ba7316 thru 0x7baea92 (0x777d bytes) 2024-05-08T03:19:49.269 INFO:teuthology.orchestra.run.smithi177.stdout:1426 writesame 0x784079a thru 0x7845e4b data_size 0x9(0x56b2 bytes) 2024-05-08T03:19:49.280 INFO:teuthology.orchestra.run.smithi177.stdout:1427 write 0xdde2ad8 thru 0xddee84d (0xbd76 bytes) 2024-05-08T03:19:49.290 INFO:teuthology.orchestra.run.smithi177.stdout:1429 trunc from 0xddee84e to 0x90cea93 2024-05-08T03:19:49.312 INFO:teuthology.orchestra.run.smithi177.stdout:1430 trunc from 0x90cea93 to 0x28072b6 2024-05-08T03:19:49.332 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:19:49.331+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e44 definitely_dead 0 2024-05-08T03:19:49.352 INFO:teuthology.orchestra.run.smithi177.stdout:1432 write 0xb1b3af5 thru 0xb1badf9 (0x7305 bytes) 2024-05-08T03:19:49.356 INFO:teuthology.orchestra.run.smithi177.stdout:1434 writesame 0x8fbc067 thru 0x8fbe237 data_size 0x1(0x21d1 bytes) 2024-05-08T03:19:49.358 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:49.358 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:49.363 INFO:teuthology.orchestra.run.smithi177.stdout:1435 write 0x6ec9d63 thru 0x6ecec1e (0x4ebc bytes) 2024-05-08T03:19:49.366 INFO:teuthology.orchestra.run.smithi177.stdout:1436 write 0x6e97f52 thru 0x6e9a703 (0x27b2 bytes) 2024-05-08T03:19:49.370 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:49.371 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:49.368+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:49.371 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.371 INFO:teuthology.orchestra.run.smithi177.stdout:1438 read 0xa257999 thru 0xa2634ad (0xbb15 bytes) 2024-05-08T03:19:49.380 INFO:teuthology.orchestra.run.smithi177.stdout:1441 write 0xd3a7276 thru 0xd3b5972 (0xe6fd bytes) 2024-05-08T03:19:49.384 INFO:teuthology.orchestra.run.smithi177.stdout:1443 punch from 0x44e969d to 0x44ecd90, (0x36f3 bytes) 2024-05-08T03:19:49.399 INFO:teuthology.orchestra.run.smithi177.stdout:1445 trunc from 0xd3b5973 to 0x136cf0e 2024-05-08T03:19:49.414 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:49.414 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:49.419 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.419 INFO:teuthology.orchestra.run.smithi177.stdout:1451 read 0x720d32 thru 0x723a06 (0x2cd5 bytes) 2024-05-08T03:19:49.434 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:49.434 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:49.443 INFO:teuthology.orchestra.run.smithi177.stdout:1452 write 0xcd66142 thru 0xcd67cdd (0x1b9c bytes) 2024-05-08T03:19:49.454 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:49.454 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:49.455 INFO:teuthology.orchestra.run.smithi177.stdout:1454 trunc from 0xcd67cde to 0x5e34cde 2024-05-08T03:19:49.470 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:49.469+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:49.475 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:49.475 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:49.477 INFO:teuthology.orchestra.run.smithi177.stdout:1455 trunc from 0x5e34cde to 0xd9a905a 2024-05-08T03:19:49.483 INFO:teuthology.orchestra.run.smithi177.stdout:1456 read 0x3c19907 thru 0x3c28a0b (0xf105 bytes) 2024-05-08T03:19:49.484 INFO:teuthology.orchestra.run.smithi177.stdout:1458 write 0x8efa01d thru 0x8efcd65 (0x2d49 bytes) 2024-05-08T03:19:49.491 INFO:teuthology.orchestra.run.smithi177.stdout:1459 write 0x23b1507 thru 0x23b55c0 (0x40ba bytes) 2024-05-08T03:19:49.491 INFO:teuthology.orchestra.run.smithi177.stdout:1461 punch from 0x53fb88c to 0x53ff8f8, (0x406c bytes) 2024-05-08T03:19:49.491 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.491 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.495 INFO:teuthology.orchestra.run.smithi177.stdout:1464 write 0xeca4c58 thru 0xecb22e2 (0xd68b bytes) 2024-05-08T03:19:49.496 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:49.496 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:49.498 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.504 INFO:teuthology.orchestra.run.smithi177.stdout:1469 trunc from 0xecb22e3 to 0x9bd3ea9 2024-05-08T03:19:49.507 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:49.507 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:49.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.507 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:20638 3.1a e44/28 3:5c28002c:::rbd_data.1182cef43756.000000000000001e:head v 44'182, mlcod=44'180)", 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.506375+0000", 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.000282026, 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030422000000000001, 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506375+0000", 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506375+0000", 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506375+0000", 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4099999999999998e-07 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506383+0000", 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0020000000000006e-06 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506394+0000", 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0849000000000001e-05 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506401+0000", 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1069999999999999e-06 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506445+0000", 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3659000000000003e-05 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.506458+0000", 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3329000000000001e-05 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:19:49.511 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:49.514 INFO:teuthology.orchestra.run.smithi177.stdout:1470 read 0x5d8ea4b thru 0x5d9ca3f (0xdff5 bytes) 2024-05-08T03:19:49.517 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:49.517 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:49.520 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:49.524 INFO:teuthology.orchestra.run.smithi177.stdout:1471 write 0xd64e97c thru 0xd651fd9 (0x365e bytes) 2024-05-08T03:19:49.534 INFO:teuthology.orchestra.run.smithi177.stdout:1474 write 0xed7ef54 thru 0xed8515a (0x6207 bytes) 2024-05-08T03:19:49.538 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:49.539 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:49.550 INFO:teuthology.orchestra.run.smithi177.stdout:1475 trunc from 0xed8515b to 0x7aea633 2024-05-08T03:19:49.559 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:49.559 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:49.571 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:49.570+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:49.574 INFO:teuthology.orchestra.run.smithi177.stdout:1476 write 0xb428a5a thru 0xb42c6e0 (0x3c87 bytes) 2024-05-08T03:19:49.577 INFO:teuthology.orchestra.run.smithi177.stdout:1477 punch from 0xb6720b to 0xb6e7b7, (0x75ac bytes) 2024-05-08T03:19:49.577 INFO:teuthology.orchestra.run.smithi177.stdout:1478 punch from 0x9edd531 to 0x9ee53bc, (0x7e8b bytes) 2024-05-08T03:19:49.577 INFO:teuthology.orchestra.run.smithi177.stdout:1479 write 0x904b6f1 thru 0x90523da (0x6cea bytes) 2024-05-08T03:19:49.580 INFO:teuthology.orchestra.run.smithi177.stdout:1480 write 0x893cdba thru 0x89416a8 (0x48ef bytes) 2024-05-08T03:19:49.583 INFO:teuthology.orchestra.run.smithi177.stdout:1481 punch from 0x6c72b22 to 0x6c73097, (0x575 bytes) 2024-05-08T03:19:49.583 INFO:teuthology.orchestra.run.smithi177.stdout:1482 read 0x8a452d1 thru 0x8a46dd1 (0x1b01 bytes) 2024-05-08T03:19:49.583 INFO:teuthology.orchestra.run.smithi177.stdout:1486 writesame 0x1407ef0 thru 0x1414762 data_size 0x5(0xc873 bytes) 2024-05-08T03:19:49.597 INFO:teuthology.orchestra.run.smithi177.stdout:1487 writesame 0xe511956 thru 0xe51957d data_size 0x3a(0x7c28 bytes) 2024-05-08T03:19:49.597 INFO:teuthology.orchestra.run.smithi177.stdout:1489 writesame 0xc98bfc4 thru 0xc99502a data_size 0x1(0x9067 bytes) 2024-05-08T03:19:49.616 INFO:teuthology.orchestra.run.smithi177.stdout:1491 read 0x3436f04 thru 0x3439433 (0x2530 bytes) 2024-05-08T03:19:49.616 INFO:teuthology.orchestra.run.smithi177.stdout:1493 write 0xad10738 thru 0xad19c69 (0x9532 bytes) 2024-05-08T03:19:49.620 INFO:teuthology.orchestra.run.smithi177.stdout:1494 read 0x16c9a1d thru 0x16c9bbf (0x1a3 bytes) 2024-05-08T03:19:49.632 INFO:teuthology.orchestra.run.smithi177.stdout:1495 trunc from 0xe51957e to 0x45a9d95 2024-05-08T03:19:49.647 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:49.647 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:49.653 INFO:teuthology.orchestra.run.smithi177.stdout:1496 read 0x1370418 thru 0x1373bf7 (0x37e0 bytes) 2024-05-08T03:19:49.654 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.658 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:49.664 INFO:teuthology.orchestra.run.smithi177.stdout:1498 writesame 0x826746d thru 0x826e333 data_size 0x17(0x6ec7 bytes) 2024-05-08T03:19:49.664 INFO:teuthology.orchestra.run.smithi177.stdout:1501 write 0x6a73f38 thru 0x6a7c886 (0x894f bytes) 2024-05-08T03:19:49.667 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.667 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.667 INFO:teuthology.orchestra.run.smithi177.stdout:1506 read 0x5c5a23b thru 0x5c623a5 (0x816b bytes) 2024-05-08T03:19:49.667 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.667 INFO:teuthology.orchestra.run.smithi177.stdout:1510 writesame 0x6f5a67a thru 0x6f69adf data_size 0xe(0xf466 bytes) 2024-05-08T03:19:49.671 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:49.670+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:49.683 INFO:teuthology.orchestra.run.smithi177.stdout:1511 write 0xdee144e thru 0xdee99c5 (0x8578 bytes) 2024-05-08T03:19:49.686 INFO:teuthology.orchestra.run.smithi177.stdout:1512 write 0x82b1dec thru 0x82bdba7 (0xbdbc bytes) 2024-05-08T03:19:49.689 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.689 INFO:teuthology.orchestra.run.smithi177.stdout:1514 write 0xc6cd63f thru 0xc6dc289 (0xec4b bytes) 2024-05-08T03:19:49.692 INFO:teuthology.orchestra.run.smithi177.stdout:1515 read 0x2df50d7 thru 0x2df9590 (0x44ba bytes) 2024-05-08T03:19:49.693 INFO:teuthology.orchestra.run.smithi177.stdout:1516 write 0x604bfa3 thru 0x6058ab3 (0xcb11 bytes) 2024-05-08T03:19:49.695 INFO:teuthology.orchestra.run.smithi177.stdout:1517 read 0x91cd67 thru 0x923bb2 (0x6e4c bytes) 2024-05-08T03:19:49.696 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.696 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.696 INFO:teuthology.orchestra.run.smithi177.stdout:1521 read 0x8a15a9c thru 0x8a21eee (0xc453 bytes) 2024-05-08T03:19:49.696 INFO:teuthology.orchestra.run.smithi177.stdout:1523 write 0xb5b87ad thru 0xb5c56ff (0xcf53 bytes) 2024-05-08T03:19:49.700 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.700 INFO:teuthology.orchestra.run.smithi177.stdout:1525 punch from 0x6c4e7f5 to 0x6c5996f, (0xb17a bytes) 2024-05-08T03:19:49.700 INFO:teuthology.orchestra.run.smithi177.stdout:1526 read 0x197d57f thru 0x1988fa0 (0xba22 bytes) 2024-05-08T03:19:49.700 INFO:teuthology.orchestra.run.smithi177.stdout:1527 read 0xd86223a thru 0xd8669a0 (0x4767 bytes) 2024-05-08T03:19:49.701 INFO:teuthology.orchestra.run.smithi177.stdout:1528 punch from 0xde1dc5f to 0xde20391, (0x2732 bytes) 2024-05-08T03:19:49.701 INFO:teuthology.orchestra.run.smithi177.stdout:1529 punch from 0xbbbc502 to 0xbbcb1e4, (0xece2 bytes) 2024-05-08T03:19:49.701 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.701 INFO:teuthology.orchestra.run.smithi177.stdout:1531 read 0x398d7fd thru 0x3996c9d (0x94a1 bytes) 2024-05-08T03:19:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:1532 read 0x632ca23 thru 0x6337655 (0xac33 bytes) 2024-05-08T03:19:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:1533 read 0x42746ff thru 0x428104c (0xc94e bytes) 2024-05-08T03:19:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:1534 punch from 0x54edc53 to 0x54fc483, (0xe830 bytes) 2024-05-08T03:19:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:1535 punch from 0xb7f80cf to 0xb80594c, (0xd87d bytes) 2024-05-08T03:19:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:1536 read 0x7209f20 thru 0x7213a57 (0x9b38 bytes) 2024-05-08T03:19:49.702 INFO:teuthology.orchestra.run.smithi177.stdout:1537 writesame 0xa8e4034 thru 0xa8e5270 data_size 0x17(0x123d bytes) 2024-05-08T03:19:49.704 INFO:teuthology.orchestra.run.smithi177.stdout:1538 writesame 0x23a57ea thru 0x23b0edb data_size 0x1(0xb6f2 bytes) 2024-05-08T03:19:49.729 INFO:teuthology.orchestra.run.smithi177.stdout:1539 write 0x553878b thru 0x553ee51 (0x66c7 bytes) 2024-05-08T03:19:49.732 INFO:teuthology.orchestra.run.smithi177.stdout:1540 read 0x6dee37f thru 0x6def9e7 (0x1669 bytes) 2024-05-08T03:19:49.736 INFO:teuthology.orchestra.run.smithi177.stdout:1541 trunc from 0xdee99c6 to 0xb41299b 2024-05-08T03:19:49.752 INFO:teuthology.orchestra.run.smithi177.stdout:1542 writesame 0x5a13913 thru 0x5a17330 data_size 0x2b(0x3a1e bytes) 2024-05-08T03:19:49.754 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.759 INFO:teuthology.orchestra.run.smithi177.stdout:1544 trunc from 0xb41299b to 0x76c10b2 2024-05-08T03:19:49.773 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:49.772+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:49.778 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.778 INFO:teuthology.orchestra.run.smithi177.stdout:1546 read 0x67587b4 thru 0x6764cf3 (0xc540 bytes) 2024-05-08T03:19:49.778 INFO:teuthology.orchestra.run.smithi177.stdout:1547 punch from 0x11eae2d to 0x11ecfb9, (0x218c bytes) 2024-05-08T03:19:49.778 INFO:teuthology.orchestra.run.smithi177.stdout:1548 read 0x26c478f thru 0x26cd062 (0x88d4 bytes) 2024-05-08T03:19:49.778 INFO:teuthology.orchestra.run.smithi177.stdout:1549 read 0x643e171 thru 0x644bffc (0xde8c bytes) 2024-05-08T03:19:49.779 INFO:teuthology.orchestra.run.smithi177.stdout:1550 write 0x6a004f8 thru 0x6a085e8 (0x80f1 bytes) 2024-05-08T03:19:49.781 INFO:teuthology.orchestra.run.smithi177.stdout:1551 punch from 0x73280b4 to 0x732fde5, (0x7d31 bytes) 2024-05-08T03:19:49.781 INFO:teuthology.orchestra.run.smithi177.stdout:1552 read 0x76a28ed thru 0x76b22c5 (0xf9d9 bytes) 2024-05-08T03:19:49.782 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.782 INFO:teuthology.orchestra.run.smithi177.stdout:1554 writesame 0x6f47721 thru 0x6f4bc94 data_size 0x1c(0x4574 bytes) 2024-05-08T03:19:49.783 INFO:teuthology.orchestra.run.smithi177.stdout:1555 write 0x191cea5 thru 0x192cc38 (0xfd94 bytes) 2024-05-08T03:19:49.787 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12442 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.580870040000001, 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036923198999999997, 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12442 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204568+0000", 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6500000000000001e-06 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207214+0000", 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0026454019999999998 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207240+0000", 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6523e-05 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207254+0000", 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.43e-05 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207335+0000", 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0395000000000005e-05 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207518+0000", 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182819 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208440+0000", 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00092235600000000004 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225462+0000", 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017022065999999999 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241319+0000", 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015856677 2024-05-08T03:19:49.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241446+0000", 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012764100000000001 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241490+0000", 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.337e-05 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12820 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.793 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127022+0000", 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.658414370999999, 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0077392310000000001, 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12820 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127022+0000", 2024-05-08T03:19:49.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127022+0000", 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127023+0000", 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.27e-07 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127025+0000", 2024-05-08T03:19:49.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.317e-06 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127031+0000", 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4139999999999999e-06 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127036+0000", 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.493e-06 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134343+0000", 2024-05-08T03:19:49.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.007306575 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134428+0000", 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5909999999999996e-05 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134762+0000", 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033309499999999999 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12821 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2490368~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.658394146999999, 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083533040000000006, 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12821 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127043+0000", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9999999999999998e-07 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127048+0000", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.339e-06 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127055+0000", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7070000000000003e-06 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127059+0000", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2660000000000003e-06 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134806+0000", 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0077463200000000001 2024-05-08T03:19:49.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.134926+0000", 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000120184 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135396+0000", 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047008800000000002 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.801 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12822 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.658370837, 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089581330000000001, 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12822 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5499999999999998e-07 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127069+0000", 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2740000000000002e-06 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127084+0000", 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5495999999999999e-05 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127093+0000", 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2779999999999999e-06 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135432+0000", 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083386629999999996 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135511+0000", 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9437000000000005e-05 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136024+0000", 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051252999999999997 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.805 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12835 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 524288~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657952448, 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092348480000000004, 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12835 2024-05-08T03:19:49.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127485+0000", 2024-05-08T03:19:49.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0500000000000002e-07 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127490+0000", 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1669999999999998e-06 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127500+0000", 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0257e-05 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127507+0000", 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4760000000000003e-06 2024-05-08T03:19:49.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136070+0000", 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.008563279 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136160+0000", 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9445999999999997e-05 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136719+0000", 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000559418 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.809 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12836 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2621440~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657901170000001, 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097351599999999996, 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12836 2024-05-08T03:19:49.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127536+0000", 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2399999999999998e-07 2024-05-08T03:19:49.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127539+0000", 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1269999999999999e-06 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127547+0000", 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4529999999999996e-06 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127552+0000", 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136745+0000", 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0091925099999999992 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136801+0000", 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6663999999999998e-05 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137271+0000", 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000469412 2024-05-08T03:19:49.813 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12851 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 655360~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657876081, 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010145049999999999, 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.814 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12851 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6199999999999998e-07 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127563+0000", 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4140000000000001e-06 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127569+0000", 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127574+0000", 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137312+0000", 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097380450000000007 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137391+0000", 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9394000000000006e-05 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137706+0000", 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031446099999999998 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12852 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2752512~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657857140000001, 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010659394000000001, 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.818 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12852 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9000000000000002e-07 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127582+0000", 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127589+0000", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5919999999999997e-06 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127592+0000", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8360000000000002e-06 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137740+0000", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010148121 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137820+0000", 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9524999999999995e-05 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138239+0000", 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00041890800000000001 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12867 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:49.822 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657839538999999, 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011192809999999999, 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12867 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127599+0000", 2024-05-08T03:19:49.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.894e-06 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127605+0000", 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127610+0000", 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8080000000000004e-06 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138262+0000", 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010652069 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138333+0000", 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0634000000000002e-05 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138790+0000", 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00045714900000000001 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12868 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2883584~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.826 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657820441, 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011717782, 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12868 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127617+0000", 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.53e-07 2024-05-08T03:19:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127619+0000", 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4870000000000001e-06 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127625+0000", 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3029999999999996e-06 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127631+0000", 2024-05-08T03:19:49.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9900000000000002e-06 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138833+0000", 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011201871 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138902+0000", 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9194000000000002e-05 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139334+0000", 2024-05-08T03:19:49.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00043158399999999999 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12883 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657798871000001, 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012139397, 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.831 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12883 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127641+0000", 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127648+0000", 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7989999999999996e-06 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127653+0000", 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139365+0000", 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01171205 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139435+0000", 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0135000000000005e-05 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139777+0000", 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034227200000000002 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12884 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3014656~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.657777502, 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012668127, 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.835 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12884 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127660+0000", 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127661+0000", 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.962e-06 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127668+0000", 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.105e-06 2024-05-08T03:19:49.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127673+0000", 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1649999999999997e-06 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139800+0000", 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012127712000000001 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139884+0000", 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3873000000000001e-05 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140327+0000", 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000442939 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12899 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1048576~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656744303, 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012150764999999999, 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.839 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12899 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:19:49.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128694+0000", 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1739999999999999e-06 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128699+0000", 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1540000000000001e-06 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128714+0000", 2024-05-08T03:19:49.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5755000000000001e-05 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128722+0000", 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9789999999999993e-06 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140360+0000", 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011637522000000001 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140439+0000", 2024-05-08T03:19:49.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9297999999999999e-05 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140843+0000", 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00040388300000000001 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12900 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3145728~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656700068999999, 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012610682999999999, 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.843 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12900 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.44e-07 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128740+0000", 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2949999999999998e-06 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128748+0000", 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.43e-06 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128753+0000", 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3399999999999997e-06 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140875+0000", 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012122198000000001 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140951+0000", 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6036000000000002e-05 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141347+0000", 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039584 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12915 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1179648~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:49.847 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656676000000001, 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013175711999999999, 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12915 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:49.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7100000000000002e-07 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128765+0000", 2024-05-08T03:19:49.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5300000000000001e-06 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128772+0000", 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7589999999999997e-06 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128778+0000", 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1000000000000003e-06 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141378+0000", 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012600668000000001 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141432+0000", 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3448000000000001e-05 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141936+0000", 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00050473599999999999 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12916 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656650989999999, 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013781595000000001, 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12916 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5900000000000002e-07 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128788+0000", 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2010000000000002e-06 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128795+0000", 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6730000000000003e-06 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128800+0000", 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3550000000000004e-06 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141966+0000", 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013165469000000001 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142011+0000", 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5331999999999999e-05 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142567+0000", 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000556106 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.855 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12931 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1310720~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656629371999999, 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014420924, 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.856 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12931 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128808+0000", 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.51e-07 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128810+0000", 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.238e-06 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128816+0000", 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128821+0000", 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2499999999999997e-06 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142597+0000", 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013775832999999999 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142679+0000", 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1757999999999999e-05 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143228+0000", 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054913200000000003 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12932 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656608797000001, 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015052658, 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12932 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9900000000000001e-07 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128830+0000", 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0999999999999998e-06 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128837+0000", 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3080000000000004e-06 2024-05-08T03:19:49.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128842+0000", 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8910000000000003e-06 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143270+0000", 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014428778999999999 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143335+0000", 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5162999999999999e-05 2024-05-08T03:19:49.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143881+0000", 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054501799999999998 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12947 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1441792~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656588728999999, 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015679150999999999, 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.864 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12947 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128850+0000", 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.013e-06 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128856+0000", 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0630000000000002e-06 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128861+0000", 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9540000000000003e-06 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143935+0000", 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015074086 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144017+0000", 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1118000000000005e-05 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144527+0000", 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051051099999999995 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12948 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3538944~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:49.868 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.656568507999999, 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016271184000000001, 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12948 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128869+0000", 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128871+0000", 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:19:49.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128877+0000", 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128882+0000", 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2719999999999997e-06 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144554+0000", 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015671378999999999 2024-05-08T03:19:49.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144626+0000", 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2265e-05 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145139+0000", 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051343000000000005 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:49.872 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:49.873 INFO:teuthology.orchestra.run.smithi177.stdout:1557 write 0xac7dea0 thru 0xac81962 (0x3ac3 bytes) 2024-05-08T03:19:49.874 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1559 write 0x2733ea3 thru 0x273c518 (0x8676 bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1560 read 0x1b20220 thru 0x1b2658d (0x636e bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1561 read 0x8d43241 thru 0x8d43d91 (0xb51 bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1562 read 0x22ee812 thru 0x22f8d7d (0xa56c bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1563 read 0x25d01a6 thru 0x25d2dc4 (0x2c1f bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1564 read 0x8ec3a8 thru 0x8f7dae (0xba07 bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1565 write 0xf64beb thru 0xf67ed1 (0x32e7 bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1569 write 0x5277e3f thru 0x52844ee (0xc6b0 bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1570 write 0x60096b5 thru 0x6009814 (0x160 bytes) 2024-05-08T03:19:49.877 INFO:teuthology.orchestra.run.smithi177.stdout:1572 trunc from 0xac81963 to 0x89479fd 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1573 read 0x255513f thru 0x256161a (0xc4dc bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1576 trunc from 0x89479fd to 0x3a9b450 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1577 punch from 0x220af7b to 0x2214536, (0x95bb bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1578 write 0xdcad1a thru 0xdd0cc4 (0x5fab bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1580 read 0xea8960 thru 0xeb8010 (0xf6b1 bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1582 write 0x35ad28a thru 0x35b9dad (0xcb24 bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1583 read 0x3a70823 thru 0x3a753b4 (0x4b92 bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1584 punch from 0x24fbad7 to 0x24fd4f8, (0x1a21 bytes) 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.878 INFO:teuthology.orchestra.run.smithi177.stdout:1586 write 0x9dfac8 thru 0x9e4163 (0x469c bytes) 2024-05-08T03:19:49.879 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:49.873+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:49.882 INFO:teuthology.orchestra.run.smithi177.stdout:1588 write 0x76e69ad thru 0x76eb2b2 (0x4906 bytes) 2024-05-08T03:19:49.898 INFO:teuthology.orchestra.run.smithi177.stdout:1589 write 0xbbeff8f thru 0xbbf1aed (0x1b5f bytes) 2024-05-08T03:19:49.899 INFO:teuthology.orchestra.run.smithi177.stdout:1590 read 0x5e19a61 thru 0x5e1a3a4 (0x944 bytes) 2024-05-08T03:19:49.899 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:49.899 INFO:teuthology.orchestra.run.smithi177.stdout:1592 read 0xa0f75b6 thru 0xa1016dd (0xa128 bytes) 2024-05-08T03:19:49.915 INFO:teuthology.orchestra.run.smithi177.stdout:1594 trunc from 0xbbf1aee to 0x259e20 2024-05-08T03:19:49.916 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:19:49.917 INFO:teuthology.orchestra.run.smithi177.stderr:marked out osd.0. 2024-05-08T03:19:49.981 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:49.980+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:50.008 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:50.009 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:50.009 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:50.009 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:50.017 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:50.102 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:50.101+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:50.163 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:50.164 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:50.172 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:50.214 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:50.212+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:50.277 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:50.276+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1496 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2609152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.158091642999999, 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015926348999999999, 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.319 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1496 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158492+0000", 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3600000000000002e-07 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158495+0000", 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8339999999999999e-06 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158497+0000", 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2230000000000001e-06 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174338+0000", 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015840481 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174376+0000", 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8642e-05 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174417+0000", 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1025000000000001e-05 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.323 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1497 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.158079753999999, 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016000936, 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1497 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3999999999999998e-07 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158504+0000", 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.071e-06 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158508+0000", 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3440000000000001e-06 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158513+0000", 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7899999999999996e-06 2024-05-08T03:19:50.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174433+0000", 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015919098999999999 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174468+0000", 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5447999999999997e-05 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174504+0000", 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5944e-05 2024-05-08T03:19:50.327 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11712 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621759751999999, 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016206332, 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11712 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694826+0000", 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1440000000000001e-06 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694833+0000", 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8839999999999994e-06 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694839+0000", 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.395e-06 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.710617+0000", 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015777899000000001 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.710705+0000", 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8290999999999999e-05 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711029+0000", 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00032433599999999999 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11713 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.332 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621737616000001, 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016660854999999999, 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11713 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.84e-07 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694847+0000", 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.46e-06 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694853+0000", 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5549999999999999e-06 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694858+0000", 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8409999999999999e-06 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711071+0000", 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016213165000000002 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711160+0000", 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8337999999999993e-05 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711506+0000", 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034621199999999998 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11714 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621719019, 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017110975, 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11714 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.337 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:50.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.53e-07 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694866+0000", 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7749999999999999e-06 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694869+0000", 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2839999999999998e-06 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694872+0000", 2024-05-08T03:19:50.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.559e-06 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711545+0000", 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016672900000000001 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711626+0000", 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1076999999999996e-05 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711975+0000", 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034912700000000002 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11715 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 884736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621708023, 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017762351999999999, 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11715 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694876+0000", 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694881+0000", 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5650000000000003e-06 2024-05-08T03:19:50.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694885+0000", 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2039999999999999e-06 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712010+0000", 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017124784000000001 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712104+0000", 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4618999999999997e-05 2024-05-08T03:19:50.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712637+0000", 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00053277399999999996 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11716 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621691741999999, 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018326299000000001, 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11716 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2000000000000001e-07 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694893+0000", 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:50.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694899+0000", 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6890000000000001e-06 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694903+0000", 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3510000000000004e-06 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712677+0000", 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017773487000000001 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712790+0000", 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011356100000000001 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713217+0000", 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042686900000000002 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11717 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621673461, 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018801206000000001, 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11717 2024-05-08T03:19:50.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694910+0000", 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694912+0000", 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.001e-06 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694917+0000", 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2510000000000002e-06 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694922+0000", 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6009999999999997e-06 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713270+0000", 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018348738999999999 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713346+0000", 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5591000000000002e-05 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713711+0000", 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00036462499999999999 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11718 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621655658, 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019351284, 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11718 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:50.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4799999999999999e-07 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694929+0000", 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8449999999999999e-06 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694935+0000", 2024-05-08T03:19:50.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5470000000000004e-06 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694940+0000", 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8099999999999997e-06 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713760+0000", 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018820130000000001 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713881+0000", 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012139800000000001 2024-05-08T03:19:50.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714278+0000", 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039720599999999999 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11719 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621637744000001, 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019846827000000001, 2024-05-08T03:19:50.358 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11719 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6399999999999998e-07 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694947+0000", 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8369999999999999e-06 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694960+0000", 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2758000000000001e-05 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694965+0000", 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7210000000000002e-06 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714344+0000", 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019379181999999998 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714442+0000", 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8158000000000004e-05 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714792+0000", 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034980699999999999 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11720 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.362 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621610909999999, 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022940031999999999, 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11720 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3799999999999998e-07 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694975+0000", 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2979999999999999e-06 2024-05-08T03:19:50.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694981+0000", 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1399999999999997e-06 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694986+0000", 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.665e-06 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714835+0000", 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019848141999999999 2024-05-08T03:19:50.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714919+0000", 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4155999999999999e-05 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.717912+0000", 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0029931929999999999 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11721 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621577631999999, 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025391021999999999, 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11721 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0499999999999999e-07 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695007+0000", 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.761e-06 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695015+0000", 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4529999999999996e-06 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695018+0000", 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7910000000000002e-06 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.717956+0000", 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022937760000000001 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.718014+0000", 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7535999999999999e-05 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720396+0000", 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.002382416 2024-05-08T03:19:50.370 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11722 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621559793999999, 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028041909, 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.371 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11722 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.48e-07 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695025+0000", 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3820000000000001e-06 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695028+0000", 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.241e-06 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695030+0000", 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4210000000000002e-06 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720442+0000", 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025412039000000001 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720499+0000", 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6721000000000001e-05 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723065+0000", 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0025658569999999999 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11723 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621549486999999, 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028402541, 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.375 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11723 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695034+0000", 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9400000000000001e-07 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695046+0000", 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2328e-05 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695057+0000", 2024-05-08T03:19:50.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0789e-05 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695062+0000", 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2909999999999998e-06 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723110+0000", 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028047704999999999 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723161+0000", 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1598000000000003e-05 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723436+0000", 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027453599999999997 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11724 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:50.379 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.621506088, 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028762263999999999, 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11724 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9499999999999998e-07 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695079+0000", 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5409999999999999e-06 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695092+0000", 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3203999999999999e-05 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695096+0000", 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2209999999999999e-06 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723482+0000", 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028385778 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723533+0000", 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0966e-05 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723839+0000", 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030625900000000002 2024-05-08T03:19:50.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:12446 3.c e33/19 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:head v 33'23, mlcod=33'21)", 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.09189333, 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016492471000000002, 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224691+0000", 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8169999999999999e-06 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229419+0000", 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0047272959999999998 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229434+0000", 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5432000000000001e-05 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229458+0000", 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3796000000000001e-05 2024-05-08T03:19:50.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229542+0000", 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4031999999999998e-05 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229559+0000", 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6820000000000002e-05 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241051+0000", 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011492005 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241182+0000", 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000131273 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12953 3.18 3:18c50615:::rbd_data.1182ca0d3624.0000000000000002:a [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.187388596, 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016630718999999999, 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12953 2024-05-08T03:19:50.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129195+0000", 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0900000000000002e-07 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129197+0000", 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3089999999999998e-06 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129205+0000", 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6389999999999992e-06 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129211+0000", 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3550000000000001e-06 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145049+0000", 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015838181 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145160+0000", 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011112600000000001 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145825+0000", 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00066469999999999995 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12954 3.18 3:18c50615:::rbd_data.1182ca0d3624.0000000000000002:a [sparse-read 4063232~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.187365095000001, 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017202562000000001, 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.393 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12954 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129224+0000", 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8640000000000001e-06 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129232+0000", 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5259999999999996e-06 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129237+0000", 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145863+0000", 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016625859999999999 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145960+0000", 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7036999999999999e-05 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146420+0000", 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00046035500000000002 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13451 3.7 3:e3f1c4ac:::rbd_data.118231ecabb9.0000000000000069:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.475234784, 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022473401, 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13451 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841350+0000", 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.545e-06 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841355+0000", 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8420000000000001e-06 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841367+0000", 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1600000000000001e-05 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841373+0000", 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4309999999999999e-06 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841433+0000", 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0028000000000002e-05 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841515+0000", 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1508000000000006e-05 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.845379+0000", 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038636790000000001 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.862654+0000", 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017274982000000001 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863725+0000", 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010711029999999999 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863777+0000", 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2753000000000001e-05 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863821+0000", 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3930000000000001e-05 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20792 3.2 3:431765fa:::rbd_data.1182cef43756.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.61193616100000003, 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024515474999999998, 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:50.403 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20792 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:50.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6199999999999996e-07 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704651+0000", 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6090000000000001e-06 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704660+0000", 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6279999999999994e-06 2024-05-08T03:19:50.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704665+0000", 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2209999999999996e-06 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704705+0000", 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9740999999999998e-05 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704754+0000", 2024-05-08T03:19:50.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9351999999999998e-05 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.708526+0000", 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037723729999999999 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.728993+0000", 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020466852000000001 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.729045+0000", 2024-05-08T03:19:50.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1307999999999997e-05 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.729113+0000", 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8853e-05 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.729162+0000", 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8675999999999999e-05 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:50.408 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:50.409 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:50.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:50.411 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:50.408+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:50.510 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:50.509+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:50.518 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:50.518 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:50.518 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:50.518 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:50.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:50.611 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:50.610+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:50.632 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:50.632 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:50.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:50.725 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:50.724+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:50.787 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:50.787 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.917399001, 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:19:50.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:50.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.793 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.917056822999999, 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:50.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:19:50.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:19:50.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:19:50.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:19:50.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.917017991000002, 2024-05-08T03:19:50.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:50.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:50.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:19:50.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.916971958000001, 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:19:50.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.810 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.915954765999999, 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:19:50.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.816 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.915918515000001, 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:50.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:19:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:19:50.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.915891215999999, 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:19:50.822 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:19:50.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:19:50.827 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.915861523, 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.828 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:19:50.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.915836007999999, 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:19:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:19:50.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:19:50.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.905721841999998, 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.838 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.841 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653382997000001, 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.845 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.847 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653364666000002, 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.851 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653340204999999, 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.852 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:19:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:19:50.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:19:50.855 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653313862000001, 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.856 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:50.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:19:50.859 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653296471000001, 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.860 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:19:50.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:19:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.863 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653279189999999, 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:19:50.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:19:50.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:19:50.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:19:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653251752999999, 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:19:50.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:50.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:19:50.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653233793999998, 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.872 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:50.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:19:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:19:50.875 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653216707999999, 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.876 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:50.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:19:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.653199724, 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:19:50.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:19:50.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:19:50.883 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.884 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.884 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.884 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:50.884 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:50.884 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:50.885 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:50.886 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:50.883+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:50.979 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:50.986 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:50.984+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:50.997 INFO:teuthology.orchestra.run.smithi177.stdout:1596 write 0x90692d5 thru 0x906e8ea (0x5616 bytes) 2024-05-08T03:19:51.012 INFO:teuthology.orchestra.run.smithi177.stdout:1597 trunc from 0x906e8eb to 0x5071558 2024-05-08T03:19:51.039 INFO:teuthology.orchestra.run.smithi177.stdout:1598 read 0x73a818 thru 0x7466b8 (0xbea1 bytes) 2024-05-08T03:19:51.039 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:51.039 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:51.039 INFO:teuthology.orchestra.run.smithi177.stdout:1604 read 0x447f1aa thru 0x448a52b (0xb382 bytes) 2024-05-08T03:19:51.055 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:51.055 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:51.055 INFO:teuthology.orchestra.run.smithi177.stdout:1605 write 0xe41ae20 thru 0xe423772 (0x8953 bytes) 2024-05-08T03:19:51.063 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:51.063 INFO:teuthology.orchestra.run.smithi177.stdout:1607 write 0x2862e4d thru 0x287101d (0xe1d1 bytes) 2024-05-08T03:19:51.065 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:51.067 INFO:teuthology.orchestra.run.smithi177.stdout:1610 read 0xadefff4 thru 0xadfd582 (0xd58f bytes) 2024-05-08T03:19:51.067 INFO:teuthology.orchestra.run.smithi177.stdout:1611 punch from 0xcfac623 to 0xcfb6102, (0x9adf bytes) 2024-05-08T03:19:51.067 INFO:teuthology.orchestra.run.smithi177.stdout:1612 read 0x3ca59c9 thru 0x3ca9b2f (0x4167 bytes) 2024-05-08T03:19:51.067 INFO:teuthology.orchestra.run.smithi177.stdout:1613 punch from 0xdc374a5 to 0xdc45120, (0xdc7b bytes) 2024-05-08T03:19:51.067 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:51.067 INFO:teuthology.orchestra.run.smithi177.stdout:1615 write 0x7862863 thru 0x78702e2 (0xda80 bytes) 2024-05-08T03:19:51.071 INFO:teuthology.orchestra.run.smithi177.stdout:1616 write 0x31e8112 thru 0x31ec846 (0x4735 bytes) 2024-05-08T03:19:51.073 INFO:teuthology.orchestra.run.smithi177.stdout:1618 write 0x9ce655d thru 0x9cead2b (0x47cf bytes) 2024-05-08T03:19:51.076 INFO:teuthology.orchestra.run.smithi177.stdout:1619 writesame 0x465ac1f thru 0x4661ee5 data_size 0x1(0x72c7 bytes) 2024-05-08T03:19:51.088 INFO:teuthology.orchestra.run.smithi177.stdout:1621 write 0xafdfa4d thru 0xafe8374 (0x8928 bytes) 2024-05-08T03:19:51.095 INFO:teuthology.orchestra.run.smithi177.stdout:1622 writesame 0x91c8325 thru 0x91d615f data_size 0x1(0xde3b bytes) 2024-05-08T03:19:51.110 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:51.109+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:51.187 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:51.186+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:51.187 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:51.188 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:51.188 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:51.188 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:51.188 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:51.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:51.288 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:51.287+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:51.344 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12443 3.1f e33/19 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head v 33'18, mlcod=32'16)", 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 15.13282152, 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032662806000000003, 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.15e-07 2024-05-08T03:19:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223558+0000", 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013815355 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223574+0000", 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6634999999999999e-05 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223595+0000", 2024-05-08T03:19:51.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0479000000000002e-05 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223664+0000", 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8885000000000002e-05 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223687+0000", 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3186000000000001e-05 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242352+0000", 2024-05-08T03:19:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018665370000000001 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242405+0000", 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2481000000000001e-05 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12446 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12582912~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 15.131306191, 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031506791999999999, 2024-05-08T03:19:51.349 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12446 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211258+0000", 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0529999999999999e-06 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222959+0000", 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011700492 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222983+0000", 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3543000000000001e-05 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222993+0000", 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.095e-05 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223072+0000", 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8232000000000006e-05 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223219+0000", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00014761700000000001 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.224068+0000", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00084822100000000002 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.241315+0000", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017247558 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:51.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242693+0000", 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001378343 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242734+0000", 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.057e-05 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242764+0000", 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0213e-05 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.354 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12923 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214460178, 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031179671999999999, 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12923 2024-05-08T03:19:51.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128104+0000", 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128106+0000", 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9889999999999999e-06 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128113+0000", 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2099999999999996e-06 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128118+0000", 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.814e-06 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.158874+0000", 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.030755981000000002 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.158961+0000", 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6848000000000006e-05 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159283+0000", 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032245499999999997 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12924 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3276800~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214439479999999, 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031663164000000001, 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.359 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12924 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128125+0000", 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9630000000000001e-06 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128134+0000", 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4680000000000003e-06 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128139+0000", 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8620000000000002e-06 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159312+0000", 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031173197999999999 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159359+0000", 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6705000000000001e-05 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159787+0000", 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00042857099999999999 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12927 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.363 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214418595, 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032219128, 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12927 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:51.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9099999999999999e-07 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128148+0000", 2024-05-08T03:19:51.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8150000000000002e-06 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128155+0000", 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0899999999999999e-06 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128160+0000", 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0440000000000003e-06 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159824+0000", 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031663320000000002 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159908+0000", 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3854e-05 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160364+0000", 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045661399999999999 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.367 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12928 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1703936~40822] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214396241999999, 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034852760000000003, 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12928 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128168+0000", 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128170+0000", 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.069e-06 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128177+0000", 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0079999999999996e-06 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999997e-06 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160383+0000", 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032201373999999998 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160454+0000", 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1266999999999994e-05 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163020+0000", 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025659490000000001 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.371 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12929 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2228224~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.21437603, 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035410024999999998, 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12929 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128190+0000", 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9769999999999999e-06 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128197+0000", 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0639999999999996e-06 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128202+0000", 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8879999999999996e-06 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163060+0000", 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034858597999999998 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163141+0000", 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0723999999999995e-05 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163598+0000", 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045642700000000003 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12930 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 3801088~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214355373, 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035878974000000001, 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12930 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128210+0000", 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.885e-06 2024-05-08T03:19:51.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128218+0000", 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0600000000000002e-06 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128222+0000", 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9169999999999997e-06 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163626+0000", 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035403794000000002 2024-05-08T03:19:51.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163692+0000", 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5364999999999995e-05 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164087+0000", 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00039560600000000001 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12939 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1310720~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.379 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214335140999999, 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.038840372999999997, 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12939 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:51.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128229+0000", 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3349999999999997e-06 2024-05-08T03:19:51.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4619999999999999e-06 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5169999999999999e-06 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164121+0000", 2024-05-08T03:19:51.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035873961000000003 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164166+0000", 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5364000000000001e-05 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167069+0000", 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00290238 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12940 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3407872~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214309825000001, 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039289147000000003, 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12940 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2899999999999999e-07 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128256+0000", 2024-05-08T03:19:51.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2249999999999999e-06 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.438e-06 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128269+0000", 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5390000000000004e-06 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167107+0000", 2024-05-08T03:19:51.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.038837572000000001 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167166+0000", 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9178999999999997e-05 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167543+0000", 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037676499999999999 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.387 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12943 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214287173000001, 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039875717999999997, 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12943 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:51.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128277+0000", 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128279+0000", 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128286+0000", 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2949999999999996e-06 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128291+0000", 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1970000000000004e-06 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167582+0000", 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039290450999999997 2024-05-08T03:19:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167631+0000", 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9113000000000002e-05 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168152+0000", 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000521132 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.391 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12944 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1835008~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214265772999999, 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040242297000000003, 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12944 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:51.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128301+0000", 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3929999999999998e-06 2024-05-08T03:19:51.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128308+0000", 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3969999999999997e-06 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.101e-06 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168167+0000", 2024-05-08T03:19:51.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039854199999999999 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168214+0000", 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6867999999999999e-05 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168540+0000", 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000325951 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12945 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2359296~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214244255000001, 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040838362000000003, 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12945 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.396 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128320+0000", 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128322+0000", 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9520000000000001e-06 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128329+0000", 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3000000000000004e-06 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7200000000000006e-06 2024-05-08T03:19:51.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168575+0000", 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040238643999999997 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168626+0000", 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0748000000000003e-05 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169158+0000", 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053160000000000002 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.399 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12946 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 3932160~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214219224000001, 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.041457121999999999, 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12946 2024-05-08T03:19:51.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128345+0000", 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.58e-07 2024-05-08T03:19:51.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128348+0000", 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128355+0000", 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4540000000000001e-06 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128360+0000", 2024-05-08T03:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7929999999999997e-06 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169204+0000", 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040844643 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169296+0000", 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1508999999999994e-05 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169802+0000", 2024-05-08T03:19:51.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00050553100000000004 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12955 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1441792~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214194459, 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044309089000000003, 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12955 2024-05-08T03:19:51.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128370+0000", 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128372+0000", 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4700000000000001e-06 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128381+0000", 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6680000000000007e-06 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128386+0000", 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4430000000000002e-06 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169824+0000", 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.041438206999999998 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169881+0000", 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6671e-05 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172678+0000", 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0027971659999999998 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12956 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3538944~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214169549999999, 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044883456000000002, 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12956 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.408 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128395+0000", 2024-05-08T03:19:51.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.46e-07 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128405+0000", 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0404000000000001e-05 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128410+0000", 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1610000000000002e-06 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128415+0000", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5600000000000004e-06 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172722+0000", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044307524000000001 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172811+0000", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8842000000000001e-05 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173278+0000", 2024-05-08T03:19:51.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046651899999999998 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12959 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214140655, 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.045215945, 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.412 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12959 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128426+0000", 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1519999999999999e-06 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128433+0000", 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9779999999999999e-06 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128437+0000", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7419999999999997e-06 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173305+0000", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044867332000000003 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173350+0000", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5664999999999997e-05 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173639+0000", 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00028864599999999999 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12960 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214119888000001, 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.046787179999999998, 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12960 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128446+0000", 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2800000000000002e-06 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128453+0000", 2024-05-08T03:19:51.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7830000000000001e-06 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128458+0000", 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4669999999999998e-06 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173666+0000", 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.045208116999999999 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173709+0000", 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3189000000000003e-05 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175231+0000", 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015219400000000001 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12961 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2490368~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.21409985, 2024-05-08T03:19:51.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047131451999999997, 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12961 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128466+0000", 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-06 2024-05-08T03:19:51.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128478+0000", 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1426e-05 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128485+0000", 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2590000000000003e-06 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175259+0000", 2024-05-08T03:19:51.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.046774064999999997 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175303+0000", 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3924000000000002e-05 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175595+0000", 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00029234399999999999 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12962 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 14.214072169, 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047620162000000001, 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12962 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128492+0000", 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9299999999999999e-07 2024-05-08T03:19:51.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128494+0000", 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4889999999999998e-06 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128502+0000", 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1999999999999997e-06 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128507+0000", 2024-05-08T03:19:51.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6459999999999998e-06 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175612+0000", 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047104641000000003 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175653+0000", 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1646000000000001e-05 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.176112+0000", 2024-05-08T03:19:51.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000458147 2024-05-08T03:19:51.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.429 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:51.429 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:51.429 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:51.430 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:51.431 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:51.429+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:51.530 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:51.530+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:51.575 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:51.576 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:51.576 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:51.576 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:51.576 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:51.584 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:51.669 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:51.668+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:51.697 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:51.698 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:51.715 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:51.715 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:51.731 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:51.731 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:51.731 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:51.731 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:51.731 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:51.732 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:51.732 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:51.732 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:51.730+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:51.739 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:51.747 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:51.748 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:51.765 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:51.765 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:51.783 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:51.783 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:51.799 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:51.800 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:51.817 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:51.817 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:51.833 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:51.832+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:51.885 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:51.885 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:51.894 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:51.933 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:51.932+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:51.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:19:52.018 INFO:teuthology.orchestra.run.smithi177.stdout:1623 write 0x44811dc thru 0x448976c (0x8591 bytes) 2024-05-08T03:19:52.021 INFO:teuthology.orchestra.run.smithi177.stdout:1625 write 0xde12de0 thru 0xde13577 (0x798 bytes) 2024-05-08T03:19:52.024 INFO:teuthology.orchestra.run.smithi177.stdout:1626 read 0xd0caf9a thru 0xd0cfb08 (0x4b6f bytes) 2024-05-08T03:19:52.025 INFO:teuthology.orchestra.run.smithi177.stdout:1628 read 0xa72d83 thru 0xa82995 (0xfc13 bytes) 2024-05-08T03:19:52.025 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.025 INFO:teuthology.orchestra.run.smithi177.stdout:1630 read 0x2767156 thru 0x276f25c (0x8107 bytes) 2024-05-08T03:19:52.026 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.026 INFO:teuthology.orchestra.run.smithi177.stdout:1633 write 0xc420fff thru 0xc4251cc (0x41ce bytes) 2024-05-08T03:19:52.027 INFO:teuthology.orchestra.run.smithi177.stdout:1635 read 0x68f7140 thru 0x68ff85c (0x871d bytes) 2024-05-08T03:19:52.027 INFO:teuthology.orchestra.run.smithi177.stdout:1636 read 0xb1b7d95 thru 0xb1c1af3 (0x9d5f bytes) 2024-05-08T03:19:52.027 INFO:teuthology.orchestra.run.smithi177.stdout:1637 writesame 0x8318d38 thru 0x831cc5e data_size 0x11(0x3f27 bytes) 2024-05-08T03:19:52.031 INFO:teuthology.orchestra.run.smithi177.stdout:1638 read 0x75679d0 thru 0x7568d06 (0x1337 bytes) 2024-05-08T03:19:52.031 INFO:teuthology.orchestra.run.smithi177.stdout:1639 punch from 0xc0f6973 to 0xc0fb040, (0x46cd bytes) 2024-05-08T03:19:52.031 INFO:teuthology.orchestra.run.smithi177.stdout:1640 trunc from 0xe423773 to 0xe25703a 2024-05-08T03:19:52.034 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:52.033+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:52.050 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.050 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.052 INFO:teuthology.orchestra.run.smithi177.stdout:1641 trunc from 0xe25703a to 0x3932d6a 2024-05-08T03:19:52.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:52.071 INFO:teuthology.orchestra.run.smithi177.stdout:1642 write 0xe35172 thru 0xe374dc (0x236b bytes) 2024-05-08T03:19:52.073 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.073 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.073 INFO:teuthology.orchestra.run.smithi177.stdout:1645 read 0x1ece344 thru 0x1ed1300 (0x2fbd bytes) 2024-05-08T03:19:52.073 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.073 INFO:teuthology.orchestra.run.smithi177.stdout:1648 write 0x24311b4 thru 0x2440fd8 (0xfe25 bytes) 2024-05-08T03:19:52.079 INFO:teuthology.orchestra.run.smithi177.stdout:1649 read 0x1a10009 thru 0x1a13ad0 (0x3ac8 bytes) 2024-05-08T03:19:52.079 INFO:teuthology.orchestra.run.smithi177.stdout:1650 read 0x3053b6 thru 0x30d2a5 (0x7ef0 bytes) 2024-05-08T03:19:52.080 INFO:teuthology.orchestra.run.smithi177.stdout:1651 punch from 0x17afcaf to 0x17b172d, (0x1a7e bytes) 2024-05-08T03:19:52.080 INFO:teuthology.orchestra.run.smithi177.stdout:1656 read 0xfef24d thru 0xffef99 (0xfd4d bytes) 2024-05-08T03:19:52.084 INFO:teuthology.orchestra.run.smithi177.stdout:1659 trunc from 0x3932d6a to 0x23fe09 2024-05-08T03:19:52.112 INFO:teuthology.orchestra.run.smithi177.stdout:1660 writesame 0xa555427 thru 0xa562272 data_size 0x6c(0xce4c bytes) 2024-05-08T03:19:52.112 INFO:teuthology.orchestra.run.smithi177.stdout:1661 read 0x705206d thru 0x70535b0 (0x1544 bytes) 2024-05-08T03:19:52.120 INFO:teuthology.orchestra.run.smithi177.stdout:1662 write 0xbc00763 thru 0xbc0b73d (0xafdb bytes) 2024-05-08T03:19:52.128 INFO:teuthology.orchestra.run.smithi177.stdout:1663 write 0xbb14f88 thru 0xbb21ac9 (0xcb42 bytes) 2024-05-08T03:19:52.133 INFO:teuthology.orchestra.run.smithi177.stdout:1664 write 0x4784d6a thru 0x4786be6 (0x1e7d bytes) 2024-05-08T03:19:52.134 INFO:teuthology.orchestra.run.smithi177.stdout:1665 read 0xf30666 thru 0xf3e45a (0xddf5 bytes) 2024-05-08T03:19:52.135 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.135 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.139 INFO:teuthology.orchestra.run.smithi177.stdout:1671 write 0xbcc5d31 thru 0xbcd3ab1 (0xdd81 bytes) 2024-05-08T03:19:52.142 INFO:teuthology.orchestra.run.smithi177.stdout:1672 writesame 0x73935a7 thru 0x739c4aa data_size 0x71(0x8f04 bytes) 2024-05-08T03:19:52.145 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:52.144+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:52.145 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.145 INFO:teuthology.orchestra.run.smithi177.stdout:1674 read 0x4f36182 thru 0x4f373db (0x125a bytes) 2024-05-08T03:19:52.145 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:52.145 INFO:teuthology.orchestra.run.smithi177.stdout:1678 read 0x752fb22 thru 0x753485f (0x4d3e bytes) 2024-05-08T03:19:52.146 INFO:teuthology.orchestra.run.smithi177.stdout:1679 read 0x2ff653c thru 0x2ff9f93 (0x3a58 bytes) 2024-05-08T03:19:52.146 INFO:teuthology.orchestra.run.smithi177.stdout:1681 punch from 0x27d443d to 0x27db057, (0x6c1a bytes) 2024-05-08T03:19:52.146 INFO:teuthology.orchestra.run.smithi177.stdout:1682 write 0x70e6762 thru 0x70f1842 (0xb0e1 bytes) 2024-05-08T03:19:52.148 INFO:teuthology.orchestra.run.smithi177.stdout:1683 read 0x82e97a0 thru 0x82f47e0 (0xb041 bytes) 2024-05-08T03:19:52.157 INFO:teuthology.orchestra.run.smithi177.stdout:1684 write 0xde7f33d thru 0xde80fbf (0x1c83 bytes) 2024-05-08T03:19:52.163 INFO:teuthology.orchestra.run.smithi177.stdout:1685 write 0x43ad906 thru 0x43ba2ab (0xc9a6 bytes) 2024-05-08T03:19:52.165 INFO:teuthology.orchestra.run.smithi177.stdout:1687 punch from 0xc65c0ca to 0xc6642e9, (0x821f bytes) 2024-05-08T03:19:52.165 INFO:teuthology.orchestra.run.smithi177.stdout:1689 write 0x8f32ba thru 0x8fcbaf (0x98f6 bytes) 2024-05-08T03:19:52.180 INFO:teuthology.orchestra.run.smithi177.stdout:1690 trunc from 0xde80fc0 to 0x2147db0 2024-05-08T03:19:52.198 INFO:teuthology.orchestra.run.smithi177.stdout:1691 writesame 0x177e29 thru 0x180822 data_size 0x57(0x89fa bytes) 2024-05-08T03:19:52.200 INFO:teuthology.orchestra.run.smithi177.stdout:1692 read 0x133e9b4 thru 0x134a993 (0xbfe0 bytes) 2024-05-08T03:19:52.210 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.210 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.216 INFO:teuthology.orchestra.run.smithi177.stdout:1693 write 0xb0f5851 thru 0xb0fa13d (0x48ed bytes) 2024-05-08T03:19:52.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:52.219 INFO:teuthology.orchestra.run.smithi177.stdout:1694 write 0x7967ab2 thru 0x796e914 (0x6e63 bytes) 2024-05-08T03:19:52.222 INFO:teuthology.orchestra.run.smithi177.stdout:1695 read 0x7f44a2f thru 0x7f5421c (0xf7ee bytes) 2024-05-08T03:19:52.223 INFO:teuthology.orchestra.run.smithi177.stdout:1697 read 0xb003a6d thru 0xb0130e9 (0xf67d bytes) 2024-05-08T03:19:52.223 INFO:teuthology.orchestra.run.smithi177.stdout:1698 writesame 0xaa5cb59 thru 0xaa65da2 data_size 0x6b(0x924a bytes) 2024-05-08T03:19:52.236 INFO:teuthology.orchestra.run.smithi177.stdout:1700 trunc from 0xb0fa13e to 0x2a26cc0 2024-05-08T03:19:52.249 INFO:teuthology.orchestra.run.smithi177.stdout:1701 read 0x1737d28 thru 0x1742ae5 (0xadbe bytes) 2024-05-08T03:19:52.260 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:52.259+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:52.264 INFO:teuthology.orchestra.run.smithi177.stdout:1702 write 0x9bdf961 thru 0x9bece8a (0xd52a bytes) 2024-05-08T03:19:52.269 INFO:teuthology.orchestra.run.smithi177.stdout:1703 clone 14 order 24 su 262144 sc 10 2024-05-08T03:19:52.337 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:52.336+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:52.364 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.364 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.373 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:52.438 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:52.437+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:52.478 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.478 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.487 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:52.538 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:52.537+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:52.592 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.592 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.601 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:52.639 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:52.638+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:52.740 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:52.739+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:52.746 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.746 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.755 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:52.841 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:52.840+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:52.860 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:52.860 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:52.869 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:52.953 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:52.951+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:52.954 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:19:53.027 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.027 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.037 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:53.043 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:53.042+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:53.124 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-05-08T03:19:53.145 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:53.144+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:53.151 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.151 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.160 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:53.247 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:53.245+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:53.269 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.270 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:53.346 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:53.344+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:53.384 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.384 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.393 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:53.476 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:53.475+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:53.537 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.538 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.546 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:53.588 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:53.586+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:53.648 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:53.647+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:53.693 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.693 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.702 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:53.749 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:53.748+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:53.829 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:53.848 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:53.848 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:53.851 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:53.850+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:53.858 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:53.952 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:53.950+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:53.959 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:53.959 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:53.979 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:53.979 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:53.997 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:53.998 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:54.015 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:54.015 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1496 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2609152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 25.859445952000002, 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015926348999999999, 2024-05-08T03:19:54.020 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1496 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158491+0000", 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158492+0000", 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3600000000000002e-07 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158495+0000", 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8339999999999999e-06 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158497+0000", 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2230000000000001e-06 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174338+0000", 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015840481 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174376+0000", 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8642e-05 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174417+0000", 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1025000000000001e-05 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.024 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1497 3.7 3:e1bd62f4:::rbd_data.11821677beed.0000000000000000:5 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 25.859434062999998, 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016000936, 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1497 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158503+0000", 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3999999999999998e-07 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158504+0000", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.071e-06 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158508+0000", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3440000000000001e-06 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.158513+0000", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7899999999999996e-06 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174433+0000", 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015919098999999999 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174468+0000", 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5447999999999997e-05 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.174504+0000", 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5944e-05 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.028 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11712 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323114060999998, 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016206332, 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11712 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694823+0000", 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694826+0000", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1440000000000001e-06 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694833+0000", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8839999999999994e-06 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694839+0000", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.395e-06 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.710617+0000", 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015777899000000001 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.710705+0000", 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8290999999999999e-05 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711029+0000", 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00032433599999999999 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.032 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11713 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323091925, 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016660854999999999, 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11713 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694845+0000", 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.84e-07 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694847+0000", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.46e-06 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694853+0000", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5549999999999999e-06 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694858+0000", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8409999999999999e-06 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711071+0000", 2024-05-08T03:19:54.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016213165000000002 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711160+0000", 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8337999999999993e-05 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711506+0000", 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034621199999999998 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11714 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323073328, 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017110975, 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11714 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694864+0000", 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.53e-07 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694866+0000", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7749999999999999e-06 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694869+0000", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2839999999999998e-06 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694872+0000", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.559e-06 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711545+0000", 2024-05-08T03:19:54.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016672900000000001 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711626+0000", 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1076999999999996e-05 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.711975+0000", 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034912700000000002 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11715 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 884736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323062331999999, 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017762351999999999, 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11715 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694875+0000", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694876+0000", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694881+0000", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5650000000000003e-06 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694885+0000", 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2039999999999999e-06 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712010+0000", 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017124784000000001 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712104+0000", 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4618999999999997e-05 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712637+0000", 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00053277399999999996 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11716 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323046050999999, 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018326299000000001, 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11716 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694891+0000", 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2000000000000001e-07 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694893+0000", 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694899+0000", 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6890000000000001e-06 2024-05-08T03:19:54.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694903+0000", 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3510000000000004e-06 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712677+0000", 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017773487000000001 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.712790+0000", 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011356100000000001 2024-05-08T03:19:54.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713217+0000", 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042686900000000002 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11717 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323027769999999, 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018801206000000001, 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.049 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11717 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694909+0000", 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694910+0000", 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694912+0000", 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.001e-06 2024-05-08T03:19:54.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694917+0000", 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2510000000000002e-06 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694922+0000", 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6009999999999997e-06 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713270+0000", 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018348738999999999 2024-05-08T03:19:54.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713346+0000", 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5591000000000002e-05 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713711+0000", 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00036462499999999999 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11718 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.053 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.323009967000001, 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019351284, 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11718 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694927+0000", 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4799999999999999e-07 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694929+0000", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8449999999999999e-06 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694935+0000", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5470000000000004e-06 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694940+0000", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8099999999999997e-06 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713760+0000", 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018820130000000001 2024-05-08T03:19:54.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.713881+0000", 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012139800000000001 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714278+0000", 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039720599999999999 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11719 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.057 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.322992053, 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019846827000000001, 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11719 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694945+0000", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6399999999999998e-07 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694947+0000", 2024-05-08T03:19:54.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8369999999999999e-06 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694960+0000", 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2758000000000001e-05 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694965+0000", 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7210000000000002e-06 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714344+0000", 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019379181999999998 2024-05-08T03:19:54.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714442+0000", 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8158000000000004e-05 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714792+0000", 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034980699999999999 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.061 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11720 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.322965219, 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022940031999999999, 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11720 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694972+0000", 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3799999999999998e-07 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694975+0000", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2979999999999999e-06 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694981+0000", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1399999999999997e-06 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.694986+0000", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.665e-06 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714835+0000", 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019848141999999999 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.714919+0000", 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4155999999999999e-05 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.717912+0000", 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0029931929999999999 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.065 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11721 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.322931941, 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025391021999999999, 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11721 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:54.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695005+0000", 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0499999999999999e-07 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695007+0000", 2024-05-08T03:19:54.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.761e-06 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695015+0000", 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4529999999999996e-06 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695018+0000", 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7910000000000002e-06 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.717956+0000", 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022937760000000001 2024-05-08T03:19:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.718014+0000", 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7535999999999999e-05 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720396+0000", 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.002382416 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11722 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:54.069 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.322914102999999, 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028041909, 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11722 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:54.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695023+0000", 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.48e-07 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695025+0000", 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3820000000000001e-06 2024-05-08T03:19:54.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695028+0000", 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.241e-06 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695030+0000", 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4210000000000002e-06 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720442+0000", 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025412039000000001 2024-05-08T03:19:54.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.720499+0000", 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6721000000000001e-05 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723065+0000", 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0025658569999999999 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.073 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11723 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.322903795999999, 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028402541, 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11723 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695033+0000", 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695034+0000", 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9400000000000001e-07 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695046+0000", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2328e-05 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695057+0000", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0789e-05 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695062+0000", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2909999999999998e-06 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723110+0000", 2024-05-08T03:19:54.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028047704999999999 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723161+0000", 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1598000000000003e-05 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723436+0000", 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027453599999999997 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.077 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:11724 3.2 3:428203b0:::rbd_data.11824ee2abc9.0000000000000005:8 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.322860396999999, 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028762263999999999, 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 11724 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695077+0000", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9499999999999998e-07 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695079+0000", 2024-05-08T03:19:54.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5409999999999999e-06 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695092+0000", 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3203999999999999e-05 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.695096+0000", 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2209999999999999e-06 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723482+0000", 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028385778 2024-05-08T03:19:54.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723533+0000", 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0966e-05 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:35.723839+0000", 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00030625900000000002 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:12446 3.c e33/19 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:head v 33'23, mlcod=33'21)", 2024-05-08T03:19:54.081 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.793247639000001, 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016492471000000002, 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224689+0000", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.224691+0000", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8169999999999999e-06 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229419+0000", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0047272959999999998 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229434+0000", 2024-05-08T03:19:54.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5432000000000001e-05 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229458+0000", 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3796000000000001e-05 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229542+0000", 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4031999999999998e-05 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.229559+0000", 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6820000000000002e-05 2024-05-08T03:19:54.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241051+0000", 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011492005 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241182+0000", 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000131273 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.085 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12953 3.18 3:18c50615:::rbd_data.1182ca0d3624.0000000000000002:a [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.888742905000001, 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016630718999999999, 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12953 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129194+0000", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129195+0000", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0900000000000002e-07 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129197+0000", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3089999999999998e-06 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129205+0000", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6389999999999992e-06 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129211+0000", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3550000000000001e-06 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145049+0000", 2024-05-08T03:19:54.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015838181 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145160+0000", 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011112600000000001 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145825+0000", 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00066469999999999995 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12954 3.18 3:18c50615:::rbd_data.1182ca0d3624.0000000000000002:a [sparse-read 4063232~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:54.089 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.888719404, 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017202562000000001, 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12954 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:54.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129218+0000", 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129224+0000", 2024-05-08T03:19:54.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8640000000000001e-06 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129232+0000", 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5259999999999996e-06 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.129237+0000", 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145863+0000", 2024-05-08T03:19:54.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016625859999999999 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145960+0000", 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7036999999999999e-05 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146420+0000", 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00046035500000000002 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.093 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13451 3.7 3:e3f1c4ac:::rbd_data.118231ecabb9.0000000000000069:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 16.176589093, 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.022473401, 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13451 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841348+0000", 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841350+0000", 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.545e-06 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841355+0000", 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8420000000000001e-06 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841367+0000", 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1600000000000001e-05 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841373+0000", 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4309999999999999e-06 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841433+0000", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0028000000000002e-05 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.841515+0000", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1508000000000006e-05 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.845379+0000", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038636790000000001 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.862654+0000", 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017274982000000001 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863725+0000", 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010711029999999999 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863777+0000", 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2753000000000001e-05 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.863821+0000", 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3930000000000001e-05 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20792 3.2 3:431765fa:::rbd_data.1182cef43756.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3132904700000001, 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024515474999999998, 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:19:54.099 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20792 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704647+0000", 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6199999999999996e-07 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704651+0000", 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6090000000000001e-06 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704660+0000", 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6279999999999994e-06 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704665+0000", 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2209999999999996e-06 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704705+0000", 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9740999999999998e-05 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.704754+0000", 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9351999999999998e-05 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.708526+0000", 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037723729999999999 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.728993+0000", 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020466852000000001 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.729045+0000", 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1307999999999997e-05 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.729113+0000", 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8853e-05 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.729162+0000", 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8675999999999999e-05 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:19:54.104 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:54.106 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:54.106 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:54.106 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:54.107 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:54.107 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:54.107 INFO:teuthology.orchestra.run.smithi177.stdout:1704 write 0xdd4a4b4 thru 0xdd4c9e2 (0x252f bytes) 2024-05-08T03:19:54.108 INFO:teuthology.orchestra.run.smithi177.stdout:1705 read 0x49608ad thru 0x496a350 (0x9aa4 bytes) 2024-05-08T03:19:54.108 INFO:teuthology.orchestra.run.smithi177.stdout:1706 trunc from 0xdd4c9e3 to 0x73c1f3e 2024-05-08T03:19:54.108 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:54.108 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:54.108 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:54.109 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:54.109 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:54.104+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:54.114 INFO:teuthology.orchestra.run.smithi177.stdout:1708 read 0x51447d1 thru 0x514e925 (0xa155 bytes) 2024-05-08T03:19:54.114 INFO:teuthology.orchestra.run.smithi177.stdout:1709 write 0xd332e5 thru 0xd38d2d (0x5a49 bytes) 2024-05-08T03:19:54.116 INFO:teuthology.orchestra.run.smithi177.stdout:1710 write 0x1360c73 thru 0x1363f4b (0x32d9 bytes) 2024-05-08T03:19:54.117 INFO:teuthology.orchestra.run.smithi177.stdout:1711 punch from 0x5d1a9a7 to 0x5d1ba98, (0x10f1 bytes) 2024-05-08T03:19:54.117 INFO:teuthology.orchestra.run.smithi177.stdout:1712 read 0x3f06114 thru 0x3f0753e (0x142b bytes) 2024-05-08T03:19:54.117 INFO:teuthology.orchestra.run.smithi177.stdout:1714 read 0x3d06f4e thru 0x3d103b9 (0x946c bytes) 2024-05-08T03:19:54.118 INFO:teuthology.orchestra.run.smithi177.stdout:1716 writesame 0x22d94fe thru 0x22dab6b data_size 0x3a(0x166e bytes) 2024-05-08T03:19:54.126 INFO:teuthology.orchestra.run.smithi177.stdout:1717 write 0x8578043 thru 0x858764f (0xf60d bytes) 2024-05-08T03:19:54.131 INFO:teuthology.orchestra.run.smithi177.stdout:1718 punch from 0x49000d3 to 0x490476e, (0x469b bytes) 2024-05-08T03:19:54.131 INFO:teuthology.orchestra.run.smithi177.stdout:1719 write 0x635a81 thru 0x63ab46 (0x50c6 bytes) 2024-05-08T03:19:54.132 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:54.143 INFO:teuthology.orchestra.run.smithi177.stdout:1722 write 0xcb1741c thru 0xcb2632e (0xef13 bytes) 2024-05-08T03:19:54.146 INFO:teuthology.orchestra.run.smithi177.stdout:1723 read 0xd3b31a thru 0xd3e744 (0x342b bytes) 2024-05-08T03:19:54.151 INFO:teuthology.orchestra.run.smithi177.stdout:1724 trunc from 0xcb2632f to 0x8533cd7 2024-05-08T03:19:54.166 INFO:teuthology.orchestra.run.smithi177.stdout:1726 writesame 0x6eac946 thru 0x6eb125c data_size 0x51(0x4917 bytes) 2024-05-08T03:19:54.167 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:19:54.167 INFO:teuthology.orchestra.run.smithi177.stdout:1728 write 0xdcb8e9 thru 0xdd383b (0x7f53 bytes) 2024-05-08T03:19:54.182 INFO:teuthology.orchestra.run.smithi177.stdout:1729 writesame 0xe4b58c4 thru 0xe4bd83a data_size 0x49(0x7f77 bytes) 2024-05-08T03:19:54.182 INFO:teuthology.orchestra.run.smithi177.stdout:1730 read 0x41507ce thru 0x4155d60 (0x5593 bytes) 2024-05-08T03:19:54.182 INFO:teuthology.orchestra.run.smithi177.stdout:1731 writesame 0x5a2b8c3 thru 0x5a36898 data_size 0x47(0xafd6 bytes) 2024-05-08T03:19:54.184 INFO:teuthology.orchestra.run.smithi177.stdout:1732 read 0x9a00a1d thru 0x9a019ec (0xfd0 bytes) 2024-05-08T03:19:54.184 INFO:teuthology.orchestra.run.smithi177.stdout:1734 read 0xbc5614c thru 0xbc56fea (0xe9f bytes) 2024-05-08T03:19:54.184 INFO:teuthology.orchestra.run.smithi177.stdout:1735 clone 15 order 20 su 262144 sc 8 2024-05-08T03:19:54.208 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:54.206+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:54.223 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:54.224 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:54.224 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:54.224 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:54.232 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:54.316 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:54.315+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:54.377 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:54.377 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:54.377 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:54.377 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:54.377 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:54.386 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:54.408 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:54.407+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:54.510 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:54.509+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:54.536 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.666461740999999, 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:19:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.666119562999999, 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:19:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.666080731000001, 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:19:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:19:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:19:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.666034698000001, 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:19:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.665017506000002, 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:19:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:19:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.664981255000001, 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:19:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:19:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:19:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.664953956000002, 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:19:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:19:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:19:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:19:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.664924263, 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:19:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:19:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.664898747999999, 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.654784582000001, 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:19:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402445737000001, 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402427406000001, 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402402944999999, 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:19:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:19:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402376602, 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:19:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402359211, 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402341929999999, 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:19:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402314492999999, 2024-05-08T03:19:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:19:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:19:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:19:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402296534000001, 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:19:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402279448000002, 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:19:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:19:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.402262464, 2024-05-08T03:19:54.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:19:54.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:19:54.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.634 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.635 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:54.635 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:54.635 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:54.636 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:54.636 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:54.635+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:54.737 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:54.735+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:54.782 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:54.782 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:54.783 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:54.783 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:54.791 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:54.837 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:54.837+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:54.897 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:54.897 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:54.897 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:54.897 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:54.897 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:54.905 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:54.938 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6] out_osds: [4, 7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:19:54.938 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:19:54.939 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [1, 2, 3, 5, 6] 2024-05-08T03:19:54.939 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-05-08T03:19:54.939 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:54.937+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:54.972 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:19:55.014 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone14 from 0xe4bd83b (overlap 0x73c1f3e) to 0x0 2024-05-08T03:19:55.039 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:55.038+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:55.059 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12443 3.1f e33/19 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head v 33'18, mlcod=32'16)", 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.847892046999998, 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032662806000000003, 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.15e-07 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223558+0000", 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013815355 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223574+0000", 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6634999999999999e-05 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223595+0000", 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0479000000000002e-05 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223664+0000", 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8885000000000002e-05 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223687+0000", 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3186000000000001e-05 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242352+0000", 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018665370000000001 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242405+0000", 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2481000000000001e-05 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12446 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12582912~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.846376717999998, 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031506791999999999, 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12446 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211258+0000", 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0529999999999999e-06 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222959+0000", 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011700492 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222983+0000", 2024-05-08T03:19:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3543000000000001e-05 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222993+0000", 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.095e-05 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223072+0000", 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8232000000000006e-05 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223219+0000", 2024-05-08T03:19:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00014761700000000001 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.224068+0000", 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00084822100000000002 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.241315+0000", 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017247558 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242693+0000", 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001378343 2024-05-08T03:19:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242734+0000", 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.057e-05 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242764+0000", 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0213e-05 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12923 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929530705000001, 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031179671999999999, 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12923 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128104+0000", 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128106+0000", 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9889999999999999e-06 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128113+0000", 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2099999999999996e-06 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128118+0000", 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.814e-06 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.158874+0000", 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.030755981000000002 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.158961+0000", 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6848000000000006e-05 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159283+0000", 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032245499999999997 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12924 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3276800~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929510007000001, 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031663164000000001, 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12924 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128125+0000", 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:19:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9630000000000001e-06 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128134+0000", 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4680000000000003e-06 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128139+0000", 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8620000000000002e-06 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159312+0000", 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031173197999999999 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159359+0000", 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6705000000000001e-05 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159787+0000", 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00042857099999999999 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12927 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929489122, 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032219128, 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12927 2024-05-08T03:19:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9099999999999999e-07 2024-05-08T03:19:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128148+0000", 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8150000000000002e-06 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128155+0000", 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0899999999999999e-06 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128160+0000", 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0440000000000003e-06 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159824+0000", 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031663320000000002 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159908+0000", 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3854e-05 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160364+0000", 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045661399999999999 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12928 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1703936~40822] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929466769000001, 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034852760000000003, 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12928 2024-05-08T03:19:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128168+0000", 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:19:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128170+0000", 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.069e-06 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128177+0000", 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0079999999999996e-06 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999997e-06 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160383+0000", 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032201373999999998 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160454+0000", 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1266999999999994e-05 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163020+0000", 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025659490000000001 2024-05-08T03:19:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12929 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2228224~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929446556999999, 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035410024999999998, 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12929 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128190+0000", 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9769999999999999e-06 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128197+0000", 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0639999999999996e-06 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128202+0000", 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8879999999999996e-06 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163060+0000", 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034858597999999998 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163141+0000", 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0723999999999995e-05 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163598+0000", 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045642700000000003 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12930 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 3801088~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929425899999998, 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035878974000000001, 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12930 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128210+0000", 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.885e-06 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128218+0000", 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0600000000000002e-06 2024-05-08T03:19:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128222+0000", 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9169999999999997e-06 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163626+0000", 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035403794000000002 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163692+0000", 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5364999999999995e-05 2024-05-08T03:19:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164087+0000", 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00039560600000000001 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12939 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1310720~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929405668000001, 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.038840372999999997, 2024-05-08T03:19:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12939 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128229+0000", 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3349999999999997e-06 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4619999999999999e-06 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5169999999999999e-06 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164121+0000", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035873961000000003 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164166+0000", 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5364000000000001e-05 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167069+0000", 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00290238 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12940 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3407872~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929380351999999, 2024-05-08T03:19:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039289147000000003, 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12940 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2899999999999999e-07 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128256+0000", 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2249999999999999e-06 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.438e-06 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128269+0000", 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5390000000000004e-06 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167107+0000", 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.038837572000000001 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167166+0000", 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9178999999999997e-05 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167543+0000", 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037676499999999999 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12943 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929357700000001, 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039875717999999997, 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12943 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128277+0000", 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128279+0000", 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:19:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128286+0000", 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2949999999999996e-06 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128291+0000", 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1970000000000004e-06 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167582+0000", 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039290450999999997 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167631+0000", 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9113000000000002e-05 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168152+0000", 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000521132 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12944 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1835008~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929336299999999, 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040242297000000003, 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12944 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128301+0000", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3929999999999998e-06 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128308+0000", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3969999999999997e-06 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.101e-06 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168167+0000", 2024-05-08T03:19:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039854199999999999 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168214+0000", 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6867999999999999e-05 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168540+0000", 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000325951 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12945 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2359296~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929314781999999, 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040838362000000003, 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12945 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128320+0000", 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128322+0000", 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9520000000000001e-06 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128329+0000", 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3000000000000004e-06 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7200000000000006e-06 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168575+0000", 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040238643999999997 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168626+0000", 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0748000000000003e-05 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169158+0000", 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053160000000000002 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.114 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12946 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 3932160~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929289750999999, 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.041457121999999999, 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12946 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.115 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128345+0000", 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.58e-07 2024-05-08T03:19:55.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128348+0000", 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128355+0000", 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4540000000000001e-06 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128360+0000", 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7929999999999997e-06 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169204+0000", 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040844643 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169296+0000", 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1508999999999994e-05 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169802+0000", 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00050553100000000004 2024-05-08T03:19:55.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12955 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1441792~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929264986, 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044309089000000003, 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12955 2024-05-08T03:19:55.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128370+0000", 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:19:55.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128372+0000", 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4700000000000001e-06 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128381+0000", 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6680000000000007e-06 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128386+0000", 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4430000000000002e-06 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169824+0000", 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.041438206999999998 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169881+0000", 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6671e-05 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172678+0000", 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0027971659999999998 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.122 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12956 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3538944~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929240076999999, 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044883456000000002, 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12956 2024-05-08T03:19:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128395+0000", 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.46e-07 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128405+0000", 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0404000000000001e-05 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128410+0000", 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1610000000000002e-06 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128415+0000", 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5600000000000004e-06 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172722+0000", 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044307524000000001 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172811+0000", 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8842000000000001e-05 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173278+0000", 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046651899999999998 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12959 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929211182, 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.045215945, 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.127 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12959 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128426+0000", 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1519999999999999e-06 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128433+0000", 2024-05-08T03:19:55.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9779999999999999e-06 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128437+0000", 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7419999999999997e-06 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173305+0000", 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044867332000000003 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173350+0000", 2024-05-08T03:19:55.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5664999999999997e-05 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173639+0000", 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00028864599999999999 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12960 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929190415000001, 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.046787179999999998, 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.131 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12960 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128446+0000", 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2800000000000002e-06 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128453+0000", 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7830000000000001e-06 2024-05-08T03:19:55.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128458+0000", 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4669999999999998e-06 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173666+0000", 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.045208116999999999 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173709+0000", 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3189000000000003e-05 2024-05-08T03:19:55.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175231+0000", 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015219400000000001 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12961 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2490368~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929170376999998, 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047131451999999997, 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.135 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12961 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128466+0000", 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-06 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128478+0000", 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1426e-05 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128485+0000", 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2590000000000003e-06 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175259+0000", 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.046774064999999997 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175303+0000", 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3924000000000002e-05 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175595+0000", 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00029234399999999999 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12962 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:55.139 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.929142696, 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047620162000000001, 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12962 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128492+0000", 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9299999999999999e-07 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128494+0000", 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4889999999999998e-06 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128502+0000", 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1999999999999997e-06 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128507+0000", 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6459999999999998e-06 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175612+0000", 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047104641000000003 2024-05-08T03:19:55.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175653+0000", 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1646000000000001e-05 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.176112+0000", 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000458147 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.143 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:55.144 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-05-08T03:19:55.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:55.146 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:55.144+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:55.245 INFO:teuthology.orchestra.run.smithi177.stdout:1736 read 0xdfcda3a thru 0xdfd26ef (0x4cb6 bytes) 2024-05-08T03:19:55.246 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:55.245+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:55.246 INFO:teuthology.orchestra.run.smithi177.stdout:1737 clone 16 order 25 su 8388608 sc 7 2024-05-08T03:19:55.269 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:19:55.268+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e50 definitely_dead 0 2024-05-08T03:19:55.290 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:55.290 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:55.290 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:55.290 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:55.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:55.346 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:55.345+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:55.443 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:55.444 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:55.444 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:55.444 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:55.444 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:55.447 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:55.446+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:55.452 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:55.548 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:55.546+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:55.558 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:55.558 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:55.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:55.558 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:55.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11826 3.a 3:5299480c:::rbd_data.1182d7df86af.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.842615107, 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.009827967, 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11826 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713659+0000", 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715527+0000", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001868562 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715543+0000", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5341000000000001e-05 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715550+0000", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4329999999999998e-06 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715605+0000", 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4818999999999998e-05 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715701+0000", 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6454999999999997e-05 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715922+0000", 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00022026700000000001 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.719967+0000", 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040455710000000004 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723322+0000", 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003355051 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723454+0000", 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013202099999999999 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723485+0000", 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0580000000000002e-05 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12065 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:head [sparse-read 0~5636096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.542473973, 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01338435, 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12065 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013800+0000", 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.53e-06 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013807+0000", 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5310000000000004e-06 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013825+0000", 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7459999999999999e-05 2024-05-08T03:19:55.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013833+0000", 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013900+0000", 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7475e-05 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013998+0000", 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405000000000007e-05 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.027182+0000", 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013184695 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.347852611, 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.568 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:19:55.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:55.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.573 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12878 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 1835008~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428666114999999, 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0095219529999999997, 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12878 2024-05-08T03:19:55.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7500000000000001e-07 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127608+0000", 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4529999999999999e-06 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127611+0000", 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2610000000000002e-06 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127615+0000", 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.481e-06 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136544+0000", 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0089289710000000008 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136650+0000", 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000105723 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137128+0000", 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000477789 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12879 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2359296~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428653568000001, 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011979432999999999, 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.578 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12879 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127619+0000", 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0200000000000001e-07 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127620+0000", 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1659999999999999e-06 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127623+0000", 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0249999999999998e-06 2024-05-08T03:19:55.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127625+0000", 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137175+0000", 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0095499769999999994 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137264+0000", 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9232000000000002e-05 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139598+0000", 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0023335299999999999 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12880 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3932160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428644126999998, 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012466181999999999, 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.582 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12880 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:55.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127629+0000", 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1060000000000001e-06 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127632+0000", 2024-05-08T03:19:55.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9569999999999998e-06 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127634+0000", 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3630000000000001e-06 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139646+0000", 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012011164 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139732+0000", 2024-05-08T03:19:55.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6537000000000002e-05 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140094+0000", 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00036183500000000002 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12893 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428228383, 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012522457000000001, 2024-05-08T03:19:55.586 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12893 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3e-07 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128047+0000", 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8310000000000001e-06 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128058+0000", 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0900000000000001e-05 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128064+0000", 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140135+0000", 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012070166 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140225+0000", 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0988000000000004e-05 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140566+0000", 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034055799999999997 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12894 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.590 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428198256000002, 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013058746, 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12894 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7800000000000002e-07 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128076+0000", 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0899999999999999e-06 2024-05-08T03:19:55.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128085+0000", 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3110000000000006e-06 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128091+0000", 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5989999999999998e-06 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140595+0000", 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012503515999999999 2024-05-08T03:19:55.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140642+0000", 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7077999999999999e-05 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141132+0000", 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049067399999999997 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.594 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428172094000001, 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:19:55.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428148022999999, 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:19:55.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428122299999998, 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:19:55.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:55.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:19:55.606 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428090582999999, 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.607 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428065479000001, 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.611 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:19:55.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:19:55.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428040674000002, 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.615 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:55.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:19:55.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.428012129999999, 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:19:55.619 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:55.623 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.427989608000001, 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:19:55.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:19:55.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.427959267999999, 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.631 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.427938087000001, 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:19:55.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:19:55.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:19:55.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.6152961509999999, 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:55.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:21947 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:10 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:54.081737+0000", 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 1.4745347, 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0092410440000000003, 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:55.642 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 21947 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081737+0000", 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081737+0000", 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081738+0000", 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2499999999999995e-07 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081743+0000", 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8149999999999996e-06 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081749+0000", 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4200000000000004e-06 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081753+0000", 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.788e-06 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.090813+0000", 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0090606000000000003 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.090867+0000", 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3739999999999997e-05 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:55.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.090978+0000", 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000111156 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:55.646 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:55.647 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:55.649 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:55.647+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:55.750 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:55.749+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:55.752 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:55.752 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:55.753 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:55.753 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:55.761 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:55.851 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:55.849+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:55.907 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:55.907 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:55.907 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:55.907 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:55.908 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:55.916 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:55.960 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:55.959+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:55.966 INFO:teuthology.orchestra.run.smithi177.stderr:marked out osd.5. 2024-05-08T03:19:55.976 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:19:56.026 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 24.150353302999999, 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.027 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808412110999999, 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808396712, 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.037 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:19:56.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:19:56.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808365070000001, 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.041 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:56.045 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808351737999999, 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:56.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.049 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808341943999999, 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:19:56.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:19:56.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808332531000001, 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.054 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:56.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:19:56.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:19:56.057 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808323009999999, 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.058 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:19:56.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808313669, 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:56.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808304454000002, 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.066 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:56.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:19:56.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:19:56.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808295092000002, 2024-05-08T03:19:56.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:56.074 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808285659999999, 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:19:56.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.078 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808276243999998, 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:56.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:19:56.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808267226000002, 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:19:56.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808258183, 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:19:56.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.090 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808245889999998, 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:56.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:19:56.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.094 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808236622999999, 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.098 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808222765, 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:56.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:19:56.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:19:56.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.808213471999998, 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 19.815199338999999, 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.110 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.111 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:56.111 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:56.111 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:56.112 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:56.111+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:56.112 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:56.114 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:19:56.213 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:56.211+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:56.225 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:56.225 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:56.225 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:56.225 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:56.234 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:56.240 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:56.240 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:56.259 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:56.259 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:56.276 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:56.276 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:56.293 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:56.293 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:56.311 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:56.311 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:56.313 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:56.312+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:56.328 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:56.329 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:56.346 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:56.347 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:56.347 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:56.357 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:19:56.363 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:56.364 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:56.414 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:56.413+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:56.468 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:56.468 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:56.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:19:56.515 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:56.514+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:56.582 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:56.582 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:56.591 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:19:56.634 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:56.633+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:56.695 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:56.695 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:56.704 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:19:56.746 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:56.744+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:56.819 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:56.818+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:56.849 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:56.850 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:56.859 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:19:56.919 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:56.918+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:56.963 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:56.963 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:56.973 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:19:57.020 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:19:57.018+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:19:57.104 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.104 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.114 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:19:57.121 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:57.120+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:57.221 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:57.220+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:57.270 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.270 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.277 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:19:57.322 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:57.321+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:57.385 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.385 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.394 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:19:57.434 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:57.433+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:57.524 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:57.523+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:57.538 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.538 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.547 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:19:57.631 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:57.629+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:57.692 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.692 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:19:57.746 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:57.744+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:57.806 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.807 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:19:57.827 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:57.825+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:57.928 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:57.927+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:57.960 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:57.960 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:57.969 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:19:57.991 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:19:58.036 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:58.034+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:58.053 INFO:teuthology.orchestra.run.smithi177.stdout:leaving image image_client.0-clone15 intact 2024-05-08T03:19:58.126 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:58.127 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:58.129 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:19:58.128+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:19:58.136 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:19:58.230 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:19:58.229+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:19:58.238 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-05-08T03:19:58.240 INFO:teuthology.orchestra.run.smithi177.stdout:1739 write 0x36fd864 thru 0x37084e1 (0xac7e bytes) 2024-05-08T03:19:58.286 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:58.286 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:58.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:19:58.337 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:58.336+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:58.376 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:19:58.383 INFO:teuthology.orchestra.run.smithi177.stdout:1740 punch from 0xb87e51c to 0xb887073, (0x8b57 bytes) 2024-05-08T03:19:58.383 INFO:teuthology.orchestra.run.smithi177.stdout:1742 writesame 0xec5185 thru 0xecc88f data_size 0x35(0x770b bytes) 2024-05-08T03:19:58.433 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:58.431+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:58.449 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:19:58.450 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:19:58.458 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:19:58.500 INFO:teuthology.orchestra.run.smithi177.stdout:1744 clone 17 order 18 su 131072 sc 2 2024-05-08T03:19:58.516 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:19:58.516 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:19:58.533 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:19:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:19:58.547 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:58.546+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:58.550 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:19:58.550 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:19:58.566 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:19:58.567 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:19:58.583 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:19:58.583 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:19:58.600 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:19:58.601 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:19:58.608 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12443 3.1f e33/19 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head v 33'18, mlcod=32'16)", 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.396613840000001, 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032662806000000003, 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.609 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209742+0000", 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.15e-07 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223558+0000", 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013815355 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223574+0000", 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6634999999999999e-05 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223595+0000", 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0479000000000002e-05 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223664+0000", 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8885000000000002e-05 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223687+0000", 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3186000000000001e-05 2024-05-08T03:19:58.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242352+0000", 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018665370000000001 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242405+0000", 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2481000000000001e-05 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12446 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12582912~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.395098511, 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031506791999999999, 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12446 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211257+0000", 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.211258+0000", 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0529999999999999e-06 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222959+0000", 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011700492 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222983+0000", 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3543000000000001e-05 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.222993+0000", 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.095e-05 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223072+0000", 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8232000000000006e-05 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.223219+0000", 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00014761700000000001 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.224068+0000", 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00084822100000000002 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.241315+0000", 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017247558 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242693+0000", 2024-05-08T03:19:58.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001378343 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242734+0000", 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.057e-05 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242764+0000", 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0213e-05 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12923 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478252498, 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031179671999999999, 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12923 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128103+0000", 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128104+0000", 2024-05-08T03:19:58.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128106+0000", 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9889999999999999e-06 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128113+0000", 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2099999999999996e-06 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128118+0000", 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.814e-06 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.158874+0000", 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.030755981000000002 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.158961+0000", 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6848000000000006e-05 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159283+0000", 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032245499999999997 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12924 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3276800~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.4782318, 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031663164000000001, 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.624 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12924 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:58.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128125+0000", 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9630000000000001e-06 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128134+0000", 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4680000000000003e-06 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128139+0000", 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8620000000000002e-06 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159312+0000", 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031173197999999999 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159359+0000", 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6705000000000001e-05 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159787+0000", 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00042857099999999999 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12927 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:58.628 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478210914999998, 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032219128, 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12927 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128145+0000", 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9099999999999999e-07 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128148+0000", 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8150000000000002e-06 2024-05-08T03:19:58.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128155+0000", 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0899999999999999e-06 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128160+0000", 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0440000000000003e-06 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159824+0000", 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031663320000000002 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.159908+0000", 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3854e-05 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160364+0000", 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045661399999999999 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12928 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1703936~40822] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.632 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478188562, 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034852760000000003, 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12928 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128167+0000", 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128168+0000", 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128170+0000", 2024-05-08T03:19:58.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.069e-06 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128177+0000", 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0079999999999996e-06 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999997e-06 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160383+0000", 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032201373999999998 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.160454+0000", 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1266999999999994e-05 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163020+0000", 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025659490000000001 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12929 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2228224~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478168350000001, 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035410024999999998, 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12929 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.637 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128188+0000", 2024-05-08T03:19:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128190+0000", 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9769999999999999e-06 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128197+0000", 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0639999999999996e-06 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128202+0000", 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8879999999999996e-06 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163060+0000", 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034858597999999998 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163141+0000", 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0723999999999995e-05 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163598+0000", 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00045642700000000003 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12930 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 3801088~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478147693, 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035878974000000001, 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.641 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12930 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128208+0000", 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128210+0000", 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.885e-06 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128218+0000", 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0600000000000002e-06 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128222+0000", 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9169999999999997e-06 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163626+0000", 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035403794000000002 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.163692+0000", 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5364999999999995e-05 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164087+0000", 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00039560600000000001 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.645 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12939 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1310720~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478127461, 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.038840372999999997, 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12939 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128228+0000", 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128229+0000", 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3349999999999997e-06 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4619999999999999e-06 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5169999999999999e-06 2024-05-08T03:19:58.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164121+0000", 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035873961000000003 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.164166+0000", 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5364000000000001e-05 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167069+0000", 2024-05-08T03:19:58.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00290238 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12940 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3407872~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478102145000001, 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039289147000000003, 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.650 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12940 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128254+0000", 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2899999999999999e-07 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128256+0000", 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2249999999999999e-06 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:19:58.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.438e-06 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128269+0000", 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5390000000000004e-06 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167107+0000", 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.038837572000000001 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167166+0000", 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9178999999999997e-05 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167543+0000", 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037676499999999999 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12943 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:58.654 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478079492999999, 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039875717999999997, 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12943 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128277+0000", 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128279+0000", 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128286+0000", 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2949999999999996e-06 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128291+0000", 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1970000000000004e-06 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167582+0000", 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039290450999999997 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.167631+0000", 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9113000000000002e-05 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168152+0000", 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000521132 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.658 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12944 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1835008~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478058093000001, 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040242297000000003, 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12944 2024-05-08T03:19:58.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128298+0000", 2024-05-08T03:19:58.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128301+0000", 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3929999999999998e-06 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128308+0000", 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3969999999999997e-06 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:58.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.101e-06 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168167+0000", 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039854199999999999 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168214+0000", 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6867999999999999e-05 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168540+0000", 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000325951 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12945 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2359296~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478036575000001, 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040838362000000003, 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12945 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128319+0000", 2024-05-08T03:19:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128320+0000", 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128322+0000", 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9520000000000001e-06 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128329+0000", 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3000000000000004e-06 2024-05-08T03:19:58.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7200000000000006e-06 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168575+0000", 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040238643999999997 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.168626+0000", 2024-05-08T03:19:58.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0748000000000003e-05 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169158+0000", 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053160000000000002 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12946 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 3932160~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.478011544000001, 2024-05-08T03:19:58.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.041457121999999999, 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12946 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:58.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128344+0000", 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128345+0000", 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.58e-07 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128348+0000", 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128355+0000", 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4540000000000001e-06 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128360+0000", 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7929999999999997e-06 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169204+0000", 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040844643 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169296+0000", 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1508999999999994e-05 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169802+0000", 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00050553100000000004 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.671 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12955 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 1441792~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.477986778999998, 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044309089000000003, 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12955 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.672 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128369+0000", 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128370+0000", 2024-05-08T03:19:58.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128372+0000", 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4700000000000001e-06 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128381+0000", 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6680000000000007e-06 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128386+0000", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4430000000000002e-06 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169824+0000", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.041438206999999998 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.169881+0000", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6671e-05 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172678+0000", 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0027971659999999998 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12956 3.c 3:367797ba:::rbd_data.1182ca0d3624.0000000000000003:a [sparse-read 3538944~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.477961870000001, 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044883456000000002, 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.676 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12956 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128394+0000", 2024-05-08T03:19:58.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128395+0000", 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.46e-07 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128405+0000", 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0404000000000001e-05 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128410+0000", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1610000000000002e-06 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128415+0000", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5600000000000004e-06 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172722+0000", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044307524000000001 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.172811+0000", 2024-05-08T03:19:58.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8842000000000001e-05 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173278+0000", 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046651899999999998 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12959 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.477932975000002, 2024-05-08T03:19:58.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.045215945, 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12959 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128423+0000", 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128426+0000", 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1519999999999999e-06 2024-05-08T03:19:58.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128433+0000", 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9779999999999999e-06 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128437+0000", 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7419999999999997e-06 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173305+0000", 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.044867332000000003 2024-05-08T03:19:58.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173350+0000", 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5664999999999997e-05 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173639+0000", 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00028864599999999999 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.684 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12960 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.477912207999999, 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.046787179999999998, 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12960 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:58.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128444+0000", 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128446+0000", 2024-05-08T03:19:58.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2800000000000002e-06 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128453+0000", 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7830000000000001e-06 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128458+0000", 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4669999999999998e-06 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173666+0000", 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.045208116999999999 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.173709+0000", 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3189000000000003e-05 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175231+0000", 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015219400000000001 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12961 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 2490368~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.477892170000001, 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047131451999999997, 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12961 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128464+0000", 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128466+0000", 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-06 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128478+0000", 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1426e-05 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128485+0000", 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2590000000000003e-06 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175259+0000", 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.046774064999999997 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175303+0000", 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3924000000000002e-05 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175595+0000", 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00029234399999999999 2024-05-08T03:19:58.692 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12962 3.c 3:36a22d6e:::rbd_data.1182ca0d3624.0000000000000005:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.477864489000002, 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047620162000000001, 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:58.693 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12962 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128491+0000", 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128492+0000", 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9299999999999999e-07 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128494+0000", 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4889999999999998e-06 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128502+0000", 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1999999999999997e-06 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:58.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128507+0000", 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6459999999999998e-06 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175612+0000", 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.047104641000000003 2024-05-08T03:19:58.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.175653+0000", 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1646000000000001e-05 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.176112+0000", 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000458147 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:58.698 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:58.699 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:58.699 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:19:58.700 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:19:58.701 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:19:58.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:19:58.702 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:19:58.702 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:58.698+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:58.800 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:19:58.799+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:19:58.846 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:58.846 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:58.846 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:58.846 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:58.855 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:19:58.901 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:58.899+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:58.958 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:58.958 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:58.959 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:58.959 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:58.959 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:58.967 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:19:58.996 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:19:59.001 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:59.000+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:59.075 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:59.075 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:59.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:59.075 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:11826 3.a 3:5299480c:::rbd_data.1182d7df86af.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.359564273, 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.009827967, 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 11826 2024-05-08T03:19:59.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713657+0000", 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.713659+0000", 2024-05-08T03:19:59.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715527+0000", 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001868562 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715543+0000", 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5341000000000001e-05 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715550+0000", 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4329999999999998e-06 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715605+0000", 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4818999999999998e-05 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715701+0000", 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6454999999999997e-05 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.715922+0000", 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00022026700000000001 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.719967+0000", 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040455710000000004 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723322+0000", 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003355051 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723454+0000", 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013202099999999999 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:35.723485+0000", 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0580000000000002e-05 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.081 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12065 3.10 3:0d0fef17:::rbd_data.11824ee2abc9.0000000000000001:head [sparse-read 0~5636096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.059423139, 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01338435, 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12065 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013798+0000", 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013800+0000", 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.53e-06 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013807+0000", 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5310000000000004e-06 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013825+0000", 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7459999999999999e-05 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013833+0000", 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013900+0000", 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7475e-05 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.013998+0000", 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405000000000007e-05 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.027182+0000", 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013184695 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.864801777, 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:19:59.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:19:59.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:19:59.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:19:59.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:19:59.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12878 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 1835008~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945615280999998, 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0095219529999999997, 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12878 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127606+0000", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7500000000000001e-07 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127608+0000", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4529999999999999e-06 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127611+0000", 2024-05-08T03:19:59.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2610000000000002e-06 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127615+0000", 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.481e-06 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136544+0000", 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0089289710000000008 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.136650+0000", 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000105723 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137128+0000", 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000477789 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12879 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2359296~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945602734000001, 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011979432999999999, 2024-05-08T03:19:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12879 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127618+0000", 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127619+0000", 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0200000000000001e-07 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127620+0000", 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1659999999999999e-06 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127623+0000", 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0249999999999998e-06 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127625+0000", 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137175+0000", 2024-05-08T03:19:59.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0095499769999999994 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.137264+0000", 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9232000000000002e-05 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139598+0000", 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0023335299999999999 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12880 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3932160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945593293000002, 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012466181999999999, 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12880 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127628+0000", 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127629+0000", 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1060000000000001e-06 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127632+0000", 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9569999999999998e-06 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.127634+0000", 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3630000000000001e-06 2024-05-08T03:19:59.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139646+0000", 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012011164 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.139732+0000", 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6537000000000002e-05 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140094+0000", 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00036183500000000002 2024-05-08T03:19:59.103 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12893 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945177549, 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012522457000000001, 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12893 2024-05-08T03:19:59.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128044+0000", 2024-05-08T03:19:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3e-07 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128047+0000", 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8310000000000001e-06 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128058+0000", 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0900000000000001e-05 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128064+0000", 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140135+0000", 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012070166 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140225+0000", 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0988000000000004e-05 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140566+0000", 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034055799999999997 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12894 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 1966080~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945147422000002, 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013058746, 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12894 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128074+0000", 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7800000000000002e-07 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128076+0000", 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0899999999999999e-06 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128085+0000", 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3110000000000006e-06 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128091+0000", 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5989999999999998e-06 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140595+0000", 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.012503515999999999 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.140642+0000", 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7077999999999999e-05 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141132+0000", 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049067399999999997 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945121260000001, 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.112 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:59.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945097188999998, 2024-05-08T03:19:59.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:19:59.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:59.120 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945071466000002, 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:19:59.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:19:59.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945039748999999, 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.128 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.945014645000001, 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.129 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:19:59.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:19:59.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.132 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.944989840000002, 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:19:59.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:19:59.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.944961295999999, 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.137 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:19:59.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.944938774000001, 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:19:59.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.944908433999998, 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.146 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:19:59.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.149 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.944887253000001, 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.150 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:19:59.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.153 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.1322453170000006, 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:19:59.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:19:59.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:59.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:19:59.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:21947 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:10 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:54.081737+0000", 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.9914838660000003, 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0092410440000000003, 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.160 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 21947 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081737+0000", 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081737+0000", 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081738+0000", 2024-05-08T03:19:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2499999999999995e-07 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081743+0000", 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8149999999999996e-06 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081749+0000", 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4200000000000004e-06 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.081753+0000", 2024-05-08T03:19:59.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.788e-06 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.090813+0000", 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0090606000000000003 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.090867+0000", 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3739999999999997e-05 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:54.090978+0000", 2024-05-08T03:19:59.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000111156 2024-05-08T03:19:59.164 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.164 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.164 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.164 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.164 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.164 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:59.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:19:59.166 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:59.164+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:59.266 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:59.265+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:59.310 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:59.310 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:59.310 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:19:59.310 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:59.319 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:19:59.367 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:19:59.365+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:19:59.463 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:59.463 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:19:59.463 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:19:59.464 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:19:59.464 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:59.468 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:59.467+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:59.472 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:19:59.569 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:19:59.568+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 27.743647157000002, 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:19:59.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:19:59.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:19:59.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401705965000001, 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.627 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401690565999999, 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:19:59.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.635 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401658923999999, 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401645592000001, 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:19:59.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.643 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401635798000001, 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.644 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:19:59.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:19:59.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:19:59.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401626385, 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.648 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:19:59.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.651 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401616864000001, 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.652 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:19:59.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401607522999999, 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.656 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401598308000001, 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.660 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:19:59.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:19:59.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401588946, 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.664 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401579514000002, 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.668 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:19:59.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:19:59.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.671 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401570098000001, 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.672 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:19:59.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:19:59.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.675 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.40156108, 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:19:59.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:19:59.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401552036999998, 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.680 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:19:59.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.683 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401539744000001, 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.684 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401530477000001, 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.688 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:19:59.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401516618999999, 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:59.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:19:59.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:19:59.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.401507326000001, 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.696 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.408493192999998, 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:19:59.700 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:19:59.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:19:59.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:19:59.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:19:59.704 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:19:59.705 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:59.704+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:59.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:19:59.806 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:19:59.805+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:19:59.815 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:59.815 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:59.815 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:19:59.815 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:59.824 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:19:59.907 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:19:59.906+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:19:59.929 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:19:59.929 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:19:59.929 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:19:59.929 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:19:59.929 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:19:59.938 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:00.024 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:00.022+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:00.064 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone16 from 0xe4bd83b (overlap 0xe4bd83b) to 0x725ec1d 2024-05-08T03:20:00.108 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:00.107+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:00.123 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:00.123 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:00.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:00.123 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:00.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.919315702999999, 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:00.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.128 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.919306252999998, 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.129 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.133 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.919293517, 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:00.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:00.139 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.919282940999999, 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:00.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:00.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.919273554, 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.144 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:00.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.919264374000001, 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.149 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:00.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688439338999999, 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.154 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:00.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:00.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.158 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688407101999999, 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:00.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688393772000001, 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.163 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:00.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.167 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688378797999999, 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.172 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688367489000001, 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688349819999999, 2024-05-08T03:20:00.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:00.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:00.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:00.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:00.183 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688334621999999, 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.184 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:00.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.688320527999998, 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:00.196 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 23.913366555, 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:00.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:00.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:00.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.201 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.990472752999999, 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:00.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:00.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.990459264999998, 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.206 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:00.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.990430283999999, 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.210 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.990403678, 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:00.214 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:00.218 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.557655416999999, 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:00.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:00.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.224 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:00.226 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:00.227 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:00.225+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:00.326 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:00.325+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:00.381 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:00.381 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:00.381 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:00.381 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:00.391 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:00.434 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:00.433+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:00.454 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-05-08T03:20:00.505 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:00.505 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:00.505 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:00.505 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:00.505 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:00.521 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:00.528 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:00.527+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:00.629 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:00.628+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13005 3.0 3:01f49529:::rbd_data.118231ecabb9.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 23.472286793999999, 2024-05-08T03:20:00.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.021952854000000001, 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13005 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205305+0000", 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205306+0000", 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0999999999999997e-07 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205310+0000", 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8249999999999998e-06 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205319+0000", 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6500000000000002e-06 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205325+0000", 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3060000000000003e-06 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205368+0000", 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2809000000000003e-05 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.205418+0000", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9920999999999999e-05 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.209135+0000", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037170160000000001 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227134+0000", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017998962 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227171+0000", 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7462999999999997e-05 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227209+0000", 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8275999999999997e-05 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.227258+0000", 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8816000000000002e-05 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.892298508, 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:00.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:00.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:00.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.745064359000001, 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.691 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:00.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:00.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:00.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20948 3.1b 3:db0960bb:::rbd_data.1182cef43756.0000000000000000:head [truncate 368160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:50.940905+0000", 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.736687517, 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.021979704999999999, 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20948 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940905+0000", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940905+0000", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940906+0000", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1289999999999999e-06 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940911+0000", 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4720000000000003e-06 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940926+0000", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4829e-05 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940934+0000", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.439e-06 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940992+0000", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7840999999999997e-05 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:00.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.944193+0000", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.003200783 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.961207+0000", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017014231000000001 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.961269+0000", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1413000000000003e-05 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.961403+0000", 2024-05-08T03:20:00.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013383199999999999 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.962497+0000", 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010945810000000001 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.962718+0000", 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000221208 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.962785+0000", 2024-05-08T03:20:00.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7088999999999994e-05 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.962884+0000", 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8857999999999994e-05 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20949 3.6 3:6626a11e:::rbd_data.1182cef43756.0000000000000001:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:50.940967+0000", 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.7366254110000003, 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028819078000000001, 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.703 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20949 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940967+0000", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940967+0000", 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940967+0000", 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5000000000000002e-07 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940971+0000", 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7110000000000001e-06 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940980+0000", 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0110000000000006e-06 2024-05-08T03:20:00.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.940987+0000", 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0199999999999997e-06 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941049+0000", 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1641000000000002e-05 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.946309+0000", 2024-05-08T03:20:00.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0052598690000000003 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.967037+0000", 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020728421 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.967162+0000", 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012488399999999999 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.967293+0000", 2024-05-08T03:20:00.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013072600000000001 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969482+0000", 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0021892930000000001 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969641+0000", 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015926300000000001 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969698+0000", 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6762000000000003e-05 2024-05-08T03:20:00.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969786+0000", 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7826999999999998e-05 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20950 3.f 3:f3d72f6c:::rbd_data.1182cef43756.0000000000000002:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:50.941213+0000", 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.7363795480000004, 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028645415, 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.709 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20950 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941213+0000", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941213+0000", 2024-05-08T03:20:00.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941213+0000", 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8100000000000002e-07 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941218+0000", 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941235+0000", 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6852999999999999e-05 2024-05-08T03:20:00.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941242+0000", 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4340000000000003e-06 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941295+0000", 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2774e-05 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.946688+0000", 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0053931700000000001 2024-05-08T03:20:00.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.967272+0000", 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020584101 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.967351+0000", 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9296999999999997e-05 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.967435+0000", 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3747999999999998e-05 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.968405+0000", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00096992300000000003 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969715+0000", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001309827 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969761+0000", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5994e-05 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969858+0000", 2024-05-08T03:20:00.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7112000000000001e-05 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20956 3.6 3:62e3555e:::rbd_data.1182cef43756.0000000000000008:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:50.941324+0000", 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.7362676910000001, 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028931990000000001, 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20956 2024-05-08T03:20:00.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941324+0000", 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941324+0000", 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941325+0000", 2024-05-08T03:20:00.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4000000000000001e-07 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941329+0000", 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8539999999999999e-06 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941337+0000", 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1960000000000008e-06 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.941344+0000", 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7309999999999996e-06 2024-05-08T03:20:00.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.946382+0000", 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0050379530000000004 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.949974+0000", 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.003592086 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.968534+0000", 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018560186999999999 2024-05-08T03:20:00.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.968601+0000", 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6837999999999999e-05 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.968715+0000", 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000114024 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.969881+0000", 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001166266 2024-05-08T03:20:00.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.970151+0000", 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00026932099999999999 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.970181+0000", 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0190000000000001e-05 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.970256+0000", 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5704000000000006e-05 2024-05-08T03:20:00.720 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23233 3.19 3:9d5f815d:::rbd_data.118254a1cd74.0000000000000008:11 [sparse-read 15728640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085420+0000", 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59217172500000004, 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017801551999999998, 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.721 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23233 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085420+0000", 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085420+0000", 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085421+0000", 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2399999999999999e-07 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085426+0000", 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5400000000000003e-06 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085430+0000", 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1160000000000001e-06 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085433+0000", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5959999999999998e-06 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.102877+0000", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017443660999999999 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103131+0000", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025484599999999998 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103222+0000", 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0569000000000006e-05 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23240 3.19 3:9bb38fd1:::rbd_data.118254a1cd74.0000000000000006:11 [sparse-read 16252928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085447+0000", 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59214529500000002, 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017953740999999999, 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.725 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23240 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085447+0000", 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085447+0000", 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085447+0000", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085448+0000", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.001e-06 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085451+0000", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7420000000000002e-06 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085456+0000", 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3820000000000003e-06 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103250+0000", 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017793526 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103322+0000", 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1842999999999996e-05 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103401+0000", 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9022000000000003e-05 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23241 3.19 3:9d5f815d:::rbd_data.118254a1cd74.0000000000000008:11 [sparse-read 15990784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085460+0000", 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59213237600000002, 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018112018000000001, 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.729 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23241 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085460+0000", 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085460+0000", 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085460+0000", 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085461+0000", 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9800000000000002e-07 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085464+0000", 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085466+0000", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.475e-06 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103426+0000", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017959711999999999 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103492+0000", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5853999999999997e-05 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103572+0000", 2024-05-08T03:20:00.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9948000000000001e-05 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23248 3.19 3:9bb38fd1:::rbd_data.118254a1cd74.0000000000000006:11 [sparse-read 16515072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085469+0000", 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59212287600000002, 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018278112999999999, 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.733 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23248 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085469+0000", 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085469+0000", 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085469+0000", 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085470+0000", 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085473+0000", 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9170000000000002e-06 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085476+0000", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0570000000000001e-06 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103598+0000", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018121157999999998 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103668+0000", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0148000000000007e-05 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103747+0000", 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9633000000000001e-05 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23249 3.19 3:9d5f815d:::rbd_data.118254a1cd74.0000000000000008:11 [sparse-read 16252928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085494+0000", 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59209779799999995, 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018426114, 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.737 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23249 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085494+0000", 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085494+0000", 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085495+0000", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4000000000000002e-07 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085497+0000", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1509999999999998e-06 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085503+0000", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4970000000000001e-06 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085507+0000", 2024-05-08T03:20:00.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6920000000000001e-06 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103773+0000", 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018265634999999999 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103842+0000", 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9030999999999997e-05 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103920+0000", 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8567999999999997e-05 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23257 3.19 3:9d5f815d:::rbd_data.118254a1cd74.0000000000000008:11 [sparse-read 16515072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085532+0000", 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59205994299999998, 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018570142000000001, 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.741 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23257 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085532+0000", 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085532+0000", 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085533+0000", 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085534+0000", 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8470000000000001e-06 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085540+0000", 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085545+0000", 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8019999999999999e-06 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.103947+0000", 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018402123999999999 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104016+0000", 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8936999999999994e-05 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104102+0000", 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6445000000000003e-05 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23263 3.19 3:9bb38fd1:::rbd_data.118254a1cd74.0000000000000006:11 [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085569+0000", 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59202285799999999, 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018711803999999999, 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.745 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23263 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085569+0000", 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085569+0000", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085570+0000", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5499999999999998e-07 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085572+0000", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.874e-06 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085577+0000", 2024-05-08T03:20:00.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8760000000000002e-06 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085582+0000", 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104129+0000", 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018546881000000001 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104199+0000", 2024-05-08T03:20:00.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9991000000000001e-05 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104281+0000", 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1944e-05 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23264 3.19 3:9d5f815d:::rbd_data.118254a1cd74.0000000000000008:11 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.085589+0000", 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59200334200000004, 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01887925, 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.749 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23264 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085589+0000", 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085589+0000", 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085589+0000", 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085610+0000", 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0479000000000002e-05 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085617+0000", 2024-05-08T03:20:00.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8809999999999998e-06 2024-05-08T03:20:00.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.085622+0000", 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0010000000000001e-06 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104307+0000", 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018685545000000001 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104383+0000", 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6322999999999994e-05 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.104468+0000", 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4641000000000005e-05 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23265 3.1a 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head [stat,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.086389+0000", 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59120278000000004, 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026560752999999999, 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.755 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23265 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086389+0000", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086389+0000", 2024-05-08T03:20:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086390+0000", 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1009999999999999e-06 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086404+0000", 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3553000000000001e-05 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086413+0000", 2024-05-08T03:20:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0850000000000003e-06 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086421+0000", 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5050000000000001e-06 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086473+0000", 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2080000000000003e-05 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086547+0000", 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4371999999999999e-05 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086729+0000", 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018221000000000001 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.111835+0000", 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025105501999999998 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.112827+0000", 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00099212600000000008 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.112891+0000", 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3870999999999999e-05 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.112950+0000", 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9348000000000002e-05 2024-05-08T03:20:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23266 3.16 3:692ca2ad:::rbd_data.118227bb2f9.0000000000000006:head [stat,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.086446+0000", 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.59114629100000005, 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.029990945000000001, 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23266 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086446+0000", 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086446+0000", 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086446+0000", 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6100000000000001e-07 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086449+0000", 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.57e-06 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086455+0000", 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8549999999999998e-06 2024-05-08T03:20:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086460+0000", 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9919999999999998e-06 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086540+0000", 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0249999999999999e-05 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086610+0000", 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0252000000000004e-05 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086804+0000", 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00019322499999999999 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.114913+0000", 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.028109417000000001 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.116343+0000", 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001430061 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.116393+0000", 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0380000000000002e-05 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.116437+0000", 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3381999999999999e-05 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23268 3.1a 3:58c57550:::rbd_object_map.118227bb2f9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.095973+0000", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.58161875799999996, 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019332588000000001, 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23268 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.095973+0000", 2024-05-08T03:20:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.095973+0000", 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.095975+0000", 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1489999999999999e-06 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.095980+0000", 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5960000000000003e-06 2024-05-08T03:20:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.095991+0000", 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0511e-05 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.095998+0000", 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.922e-06 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.110699+0000", 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014701163 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.110831+0000", 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013188699999999999 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.111283+0000", 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00045237700000000001 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.114853+0000", 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0035703699999999998 2024-05-08T03:20:00.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.115217+0000", 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00036352999999999998 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.115270+0000", 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3436000000000001e-05 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.115306+0000", 2024-05-08T03:20:00.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5646999999999999e-05 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.55492419199999998, 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.772 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.777 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.53018364600000001, 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:00.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:00.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:00.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.782 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.783 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.783 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.783 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:00.783 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:00.785 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:00.785 INFO:teuthology.orchestra.run.smithi177.stdout:1745 read 0x852e23f thru 0x853c1af (0xdf71 bytes) 2024-05-08T03:20:00.785 INFO:teuthology.orchestra.run.smithi177.stdout:1747 writesame 0x441fe6c thru 0x4426f08 data_size 0x1(0x709d bytes) 2024-05-08T03:20:00.786 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:00.783+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:00.805 INFO:teuthology.orchestra.run.smithi177.stdout:1748 read 0x972f31f thru 0x973b81a (0xc4fc bytes) 2024-05-08T03:20:00.807 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.807 INFO:teuthology.orchestra.run.smithi177.stdout:1750 write 0x9c2274b thru 0x9c2d363 (0xac19 bytes) 2024-05-08T03:20:00.810 INFO:teuthology.orchestra.run.smithi177.stdout:1752 read 0x7d63f95 thru 0x7d68d05 (0x4d71 bytes) 2024-05-08T03:20:00.811 INFO:teuthology.orchestra.run.smithi177.stdout:1753 punch from 0x8c409ea to 0x8c48052, (0x7668 bytes) 2024-05-08T03:20:00.811 INFO:teuthology.orchestra.run.smithi177.stdout:1754 write 0x928e391 thru 0x9296868 (0x84d8 bytes) 2024-05-08T03:20:00.816 INFO:teuthology.orchestra.run.smithi177.stdout:1755 read 0x35ccf54 thru 0x35cfbb9 (0x2c66 bytes) 2024-05-08T03:20:00.817 INFO:teuthology.orchestra.run.smithi177.stdout:1757 write 0x9f10c2 thru 0x9f3e2f (0x2d6e bytes) 2024-05-08T03:20:00.821 INFO:teuthology.orchestra.run.smithi177.stdout:1759 punch from 0xa259a4b to 0xa261632, (0x7be7 bytes) 2024-05-08T03:20:00.822 INFO:teuthology.orchestra.run.smithi177.stdout:1760 read 0x9e2584f thru 0x9e269b8 (0x116a bytes) 2024-05-08T03:20:00.822 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.822 INFO:teuthology.orchestra.run.smithi177.stdout:1765 punch from 0x66c5f9 to 0x67b50d, (0xef14 bytes) 2024-05-08T03:20:00.822 INFO:teuthology.orchestra.run.smithi177.stdout:1766 read 0x3f5bfc5 thru 0x3f5e13b (0x2177 bytes) 2024-05-08T03:20:00.823 INFO:teuthology.orchestra.run.smithi177.stdout:1767 punch from 0xbaf231e to 0xbaf98a6, (0x7588 bytes) 2024-05-08T03:20:00.823 INFO:teuthology.orchestra.run.smithi177.stdout:1768 punch from 0x47e0caa to 0x47eec00, (0xdf56 bytes) 2024-05-08T03:20:00.823 INFO:teuthology.orchestra.run.smithi177.stdout:1772 read 0xaef76f0 thru 0xaef8f0d (0x181e bytes) 2024-05-08T03:20:00.823 INFO:teuthology.orchestra.run.smithi177.stdout:1774 write 0x188fabc thru 0x189f4d5 (0xfa1a bytes) 2024-05-08T03:20:00.826 INFO:teuthology.orchestra.run.smithi177.stdout:1775 read 0xa9581dc thru 0xa966fc6 (0xedeb bytes) 2024-05-08T03:20:00.826 INFO:teuthology.orchestra.run.smithi177.stdout:1776 write 0x625cee3 thru 0x6262dad (0x5ecb bytes) 2024-05-08T03:20:00.831 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.833 INFO:teuthology.orchestra.run.smithi177.stdout:1779 trunc from 0xe4bd83b to 0xc8677eb 2024-05-08T03:20:00.842 INFO:teuthology.orchestra.run.smithi177.stdout:1780 writesame 0x423087a thru 0x4235256 data_size 0x21(0x49dd bytes) 2024-05-08T03:20:00.848 INFO:teuthology.orchestra.run.smithi177.stdout:1781 write 0x2658303 thru 0x265d1bb (0x4eb9 bytes) 2024-05-08T03:20:00.852 INFO:teuthology.orchestra.run.smithi177.stdout:1782 read 0xb00b96b thru 0xb018403 (0xca99 bytes) 2024-05-08T03:20:00.852 INFO:teuthology.orchestra.run.smithi177.stdout:1784 write 0x6524f3e thru 0x652806e (0x3131 bytes) 2024-05-08T03:20:00.856 INFO:teuthology.orchestra.run.smithi177.stdout:1785 punch from 0x339b6a5 to 0x33a3314, (0x7c6f bytes) 2024-05-08T03:20:00.857 INFO:teuthology.orchestra.run.smithi177.stdout:1786 punch from 0x3f77223 to 0x3f7de36, (0x6c13 bytes) 2024-05-08T03:20:00.861 INFO:teuthology.orchestra.run.smithi177.stdout:1787 trunc from 0xc8677eb to 0x91efd70 2024-05-08T03:20:00.885 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:00.884+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:00.887 INFO:teuthology.orchestra.run.smithi177.stdout:1788 writesame 0xd0be705 thru 0xd0c7bdc data_size 0x16(0x94d8 bytes) 2024-05-08T03:20:00.887 INFO:teuthology.orchestra.run.smithi177.stdout:1789 write 0xa6aeb36 thru 0xa6b0fc6 (0x2491 bytes) 2024-05-08T03:20:00.892 INFO:teuthology.orchestra.run.smithi177.stdout:1791 writesame 0x6266b11 thru 0x62753ad data_size 0x2f(0xe89d bytes) 2024-05-08T03:20:00.894 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.894 INFO:teuthology.orchestra.run.smithi177.stdout:1794 write 0x4d8a4c9 thru 0x4d955cc (0xb104 bytes) 2024-05-08T03:20:00.899 INFO:teuthology.orchestra.run.smithi177.stdout:1795 write 0x18a459d thru 0x18a6476 (0x1eda bytes) 2024-05-08T03:20:00.903 INFO:teuthology.orchestra.run.smithi177.stdout:1796 write 0x8f5cd7d thru 0x8f5f248 (0x24cc bytes) 2024-05-08T03:20:00.909 INFO:teuthology.orchestra.run.smithi177.stdout:1797 writesame 0xc4dbf8e thru 0xc4e85f5 data_size 0x1c(0xc668 bytes) 2024-05-08T03:20:00.914 INFO:teuthology.orchestra.run.smithi177.stdout:1799 read 0xbbeccd8 thru 0xbbf7f30 (0xb259 bytes) 2024-05-08T03:20:00.914 INFO:teuthology.orchestra.run.smithi177.stdout:1801 punch from 0x4afe291 to 0x4b00313, (0x2082 bytes) 2024-05-08T03:20:00.915 INFO:teuthology.orchestra.run.smithi177.stdout:1802 read 0x23d384c thru 0x23e238c (0xeb41 bytes) 2024-05-08T03:20:00.915 INFO:teuthology.orchestra.run.smithi177.stdout:1803 write 0x471954f thru 0x47238d2 (0xa384 bytes) 2024-05-08T03:20:00.923 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.924 INFO:teuthology.orchestra.run.smithi177.stdout:1806 write 0x896cd8d thru 0x8971406 (0x467a bytes) 2024-05-08T03:20:00.928 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.928 INFO:teuthology.orchestra.run.smithi177.stdout:1808 writesame 0x74fbdd3 thru 0x750b339 data_size 0x2b(0xf567 bytes) 2024-05-08T03:20:00.935 INFO:teuthology.orchestra.run.smithi177.stdout:1809 trunc from 0xd0c7bdd to 0xe75fdf5 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:24419 3.1f 3:ff6966e6:::rbd_header.1182bea5217b:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.936242+0000", 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00017061800000000001, 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000201263, 2024-05-08T03:20:00.937 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 24419 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936242+0000", 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936242+0000", 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936243+0000", 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.809e-06 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936252+0000", 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3650000000000004e-06 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936267+0000", 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5727e-05 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936275+0000", 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5700000000000004e-06 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936327+0000", 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2157999999999998e-05 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:00.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.936388+0000", 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0406999999999999e-05 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:20:00.941 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:00.947 INFO:teuthology.orchestra.run.smithi177.stdout:1810 write 0xee122a9 thru 0xee1fc8b (0xd9e3 bytes) 2024-05-08T03:20:00.949 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:00.951 INFO:teuthology.orchestra.run.smithi177.stdout:1812 write 0xa280a6a thru 0xa28b880 (0xae17 bytes) 2024-05-08T03:20:00.953 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:00.953 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:00.954 INFO:teuthology.orchestra.run.smithi177.stdout:1813 read 0x1b69918 thru 0x1b6e97e (0x5067 bytes) 2024-05-08T03:20:00.959 INFO:teuthology.orchestra.run.smithi177.stdout:1814 trunc from 0xee1fc8c to 0xacea685 2024-05-08T03:20:00.974 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:00.974 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:00.982 INFO:teuthology.orchestra.run.smithi177.stdout:1815 writesame 0xb3cbc96 thru 0xb3d873d data_size 0x14(0xcaa8 bytes) 2024-05-08T03:20:00.982 INFO:teuthology.orchestra.run.smithi177.stdout:1816 writesame 0x7d7ebe5 thru 0x7d84545 data_size 0x1d(0x5961 bytes) 2024-05-08T03:20:00.986 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:00.984+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:00.987 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6] out_osds: [4, 7, 0, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:20:00.987 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:00.988 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T03:20:00.988 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-05-08T03:20:00.988 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2024-05-08T03:20:00.990 INFO:teuthology.orchestra.run.smithi177.stdout:1817 read 0x5490140 thru 0x549d8ae (0xd76f bytes) 2024-05-08T03:20:00.991 INFO:teuthology.orchestra.run.smithi177.stdout:1818 write 0x900c3e7 thru 0x9013bc4 (0x77de bytes) 2024-05-08T03:20:00.995 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:00.995 INFO:teuthology.orchestra.run.smithi177.stdout:1820 read 0x6046355 thru 0x6049c12 (0x38be bytes) 2024-05-08T03:20:00.995 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:00.996 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:00.996 INFO:teuthology.orchestra.run.smithi177.stdout:1821 read 0x5c872b2 thru 0x5c9714e (0xfe9d bytes) 2024-05-08T03:20:00.996 INFO:teuthology.orchestra.run.smithi177.stdout:1823 punch from 0x6449c8e to 0x6451552, (0x78c4 bytes) 2024-05-08T03:20:01.004 INFO:teuthology.orchestra.run.smithi177.stdout:1824 write 0xc69d4f1 thru 0xc6a4d26 (0x7836 bytes) 2024-05-08T03:20:01.012 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:20:01.015 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.015 INFO:teuthology.orchestra.run.smithi177.stdout:1827 read 0xb994b7a thru 0xb99e946 (0x9dcd bytes) 2024-05-08T03:20:01.015 INFO:teuthology.orchestra.run.smithi177.stdout:1828 read 0xaeedad2 thru 0xaefaba8 (0xd0d7 bytes) 2024-05-08T03:20:01.015 INFO:teuthology.orchestra.run.smithi177.stdout:1829 punch from 0x53e5f57 to 0x53ea650, (0x46f9 bytes) 2024-05-08T03:20:01.015 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.015 INFO:teuthology.orchestra.run.smithi177.stdout:1832 write 0x19c9bc5 thru 0x19d4aa8 (0xaee4 bytes) 2024-05-08T03:20:01.019 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.019 INFO:teuthology.orchestra.run.smithi177.stdout:1834 punch from 0xb2bcf5a to 0xb2bd7f6, (0x89c bytes) 2024-05-08T03:20:01.019 INFO:teuthology.orchestra.run.smithi177.stdout:1837 write 0xc6ddcc thru 0xc722c5 (0x44fa bytes) 2024-05-08T03:20:01.024 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:01.024 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:01.028 INFO:teuthology.orchestra.run.smithi177.stdout:1838 trunc from 0xc6a4d27 to 0x951862c 2024-05-08T03:20:01.042 INFO:teuthology.orchestra.run.smithi177.stdout:1839 punch from 0x550de60 to 0x5518b58, (0xacf8 bytes) 2024-05-08T03:20:01.042 INFO:teuthology.orchestra.run.smithi177.stdout:1842 read 0x5b51553 thru 0x5b61255 (0xfd03 bytes) 2024-05-08T03:20:01.042 INFO:teuthology.orchestra.run.smithi177.stdout:1843 read 0x13f723e thru 0x13f84f1 (0x12b4 bytes) 2024-05-08T03:20:01.043 INFO:teuthology.orchestra.run.smithi177.stdout:1845 trunc from 0x951862c to 0x9dd71a1 2024-05-08T03:20:01.049 INFO:teuthology.orchestra.run.smithi177.stdout:1846 write 0x5bda5aa thru 0x5be9ccd (0xf724 bytes) 2024-05-08T03:20:01.058 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:01.058 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:01.067 INFO:teuthology.orchestra.run.smithi177.stdout:1847 writesame 0xd1872fd thru 0xd195875 data_size 0x1f(0xe579 bytes) 2024-05-08T03:20:01.067 INFO:teuthology.orchestra.run.smithi177.stdout:1848 write 0x1f54e8e thru 0x1f5c90c (0x7a7f bytes) 2024-05-08T03:20:01.071 INFO:teuthology.orchestra.run.smithi177.stdout:1849 writesame 0xa93c58b thru 0xa93d5e4 data_size 0x1(0x105a bytes) 2024-05-08T03:20:01.075 INFO:teuthology.orchestra.run.smithi177.stdout:1850 write 0x9c5c5c4 thru 0x9c6061a (0x4057 bytes) 2024-05-08T03:20:01.080 INFO:teuthology.orchestra.run.smithi177.stdout:1851 read 0xcf3a342 thru 0xcf3f48f (0x514e bytes) 2024-05-08T03:20:01.080 INFO:teuthology.orchestra.run.smithi177.stdout:1852 read 0x63c4a71 thru 0x63cd85e (0x8dee bytes) 2024-05-08T03:20:01.080 INFO:teuthology.orchestra.run.smithi177.stdout:1853 write 0x7ec6cbe thru 0x7ec9cc2 (0x3005 bytes) 2024-05-08T03:20:01.086 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.086 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.086 INFO:teuthology.orchestra.run.smithi177.stdout:1856 read 0xa023364 thru 0xa02da4b (0xa6e8 bytes) 2024-05-08T03:20:01.087 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:01.085+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:01.091 INFO:teuthology.orchestra.run.smithi177.stdout:1858 trunc from 0xd195876 to 0x931eb4e 2024-05-08T03:20:01.091 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.091 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.092 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:20:01.093 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:20:01.103 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:01.105 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.105 INFO:teuthology.orchestra.run.smithi177.stdout:1860 writesame 0x6e5cdc8 thru 0x6e6c24e data_size 0x1(0xf487 bytes) 2024-05-08T03:20:01.127 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:01.127 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:01.134 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.134 INFO:teuthology.orchestra.run.smithi177.stdout:1862 read 0x4273fa5 thru 0x427c63f (0x869b bytes) 2024-05-08T03:20:01.141 INFO:teuthology.orchestra.run.smithi177.stdout:1863 trunc from 0x931eb4e to 0x3e7662d 2024-05-08T03:20:01.150 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:01.150 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:01.179 INFO:teuthology.orchestra.run.smithi177.stdout:1864 trunc from 0x3e7662d to 0xb79672e 2024-05-08T03:20:01.187 INFO:teuthology.orchestra.run.smithi177.stdout:1865 trunc from 0xb79672e to 0xa07be07 2024-05-08T03:20:01.187 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:01.186+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:01.193 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.194 INFO:teuthology.orchestra.run.smithi177.stdout:1867 read 0x6991c53 thru 0x6992d11 (0x10bf bytes) 2024-05-08T03:20:01.194 INFO:teuthology.orchestra.run.smithi177.stdout:1869 read 0x90d01f1 thru 0x90d74fe (0x730e bytes) 2024-05-08T03:20:01.194 INFO:teuthology.orchestra.run.smithi177.stdout:1870 read 0x96a56a8 thru 0x96acc3b (0x7594 bytes) 2024-05-08T03:20:01.194 INFO:teuthology.orchestra.run.smithi177.stdout:1871 punch from 0x87761de to 0x8776884, (0x6a6 bytes) 2024-05-08T03:20:01.194 INFO:teuthology.orchestra.run.smithi177.stdout:1872 read 0x92decbc thru 0x92ecc3c (0xdf81 bytes) 2024-05-08T03:20:01.201 INFO:teuthology.orchestra.run.smithi177.stdout:1873 trunc from 0xa07be07 to 0x3c50bb1 2024-05-08T03:20:01.231 INFO:teuthology.orchestra.run.smithi177.stdout:1875 writesame 0xc35bcb6 thru 0xc36345d data_size 0xe(0x77a8 bytes) 2024-05-08T03:20:01.232 INFO:teuthology.orchestra.run.smithi177.stdout:1876 writesame 0xbffae75 thru 0xbfff3c0 data_size 0xa(0x454c bytes) 2024-05-08T03:20:01.235 INFO:teuthology.orchestra.run.smithi177.stdout:1877 write 0x9bfe26e thru 0x9c0c2a4 (0xe037 bytes) 2024-05-08T03:20:01.243 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.243 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.249 INFO:teuthology.orchestra.run.smithi177.stdout:1879 trunc from 0xc36345e to 0x3e8f542 2024-05-08T03:20:01.257 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:01.260 INFO:teuthology.orchestra.run.smithi177.stdout:1881 read 0x353e023 thru 0x35409bf (0x299d bytes) 2024-05-08T03:20:01.263 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.273 INFO:teuthology.orchestra.run.smithi177.stdout:1883 write 0x8ae0bf5 thru 0x8aefe9a (0xf2a6 bytes) 2024-05-08T03:20:01.276 INFO:teuthology.orchestra.run.smithi177.stdout:1884 write 0xf90102 thru 0xf90776 (0x675 bytes) 2024-05-08T03:20:01.281 INFO:teuthology.orchestra.run.smithi177.stdout:1885 write 0x2c31ce0 thru 0x2c3f355 (0xd676 bytes) 2024-05-08T03:20:01.285 INFO:teuthology.orchestra.run.smithi177.stdout:1887 read 0x6ee07dc thru 0x6eec237 (0xba5c bytes) 2024-05-08T03:20:01.285 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:01.288 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:01.287+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:01.290 INFO:teuthology.orchestra.run.smithi177.stdout:1889 trunc from 0x8aefe9b to 0x4c0ccd5 2024-05-08T03:20:01.310 INFO:teuthology.orchestra.run.smithi177.stdout:1890 write 0x4dc1391 thru 0x4dc2554 (0x11c4 bytes) 2024-05-08T03:20:01.313 INFO:teuthology.orchestra.run.smithi177.stdout:1891 read 0x22ae814 thru 0x22b8473 (0x9c60 bytes) 2024-05-08T03:20:01.314 INFO:teuthology.orchestra.run.smithi177.stdout:1893 write 0x3807976 thru 0x381681e (0xeea9 bytes) 2024-05-08T03:20:01.318 INFO:teuthology.orchestra.run.smithi177.stdout:1894 read 0x1563b76 thru 0x1571c8a (0xe115 bytes) 2024-05-08T03:20:01.319 INFO:teuthology.orchestra.run.smithi177.stdout:1896 punch from 0x1600b1e to 0x160570d, (0x4bef bytes) 2024-05-08T03:20:01.320 INFO:teuthology.orchestra.run.smithi177.stdout:1897 read 0x34ba17b thru 0x34c120b (0x7091 bytes) 2024-05-08T03:20:01.320 INFO:teuthology.orchestra.run.smithi177.stdout:1899 clone 18 order 18 su 4096 sc 6 2024-05-08T03:20:01.377 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.378 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.387 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:01.389 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:01.388+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:01.490 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:01.489+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:01.531 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.531 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.540 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:01.591 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:01.590+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:01.646 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.646 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.654 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:01.699 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:01.698+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:01.764 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.764 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:01.815 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:01.814+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:01.893 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:01.892+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:01.917 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:01.917 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:01.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:01.994 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:01.993+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:02.008 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.6 primary-affinity to 0 (802) 2024-05-08T03:20:02.012 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:20:02.035 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.035 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:20:02.095 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:02.094+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:02.189 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.189 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.196 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:02.195+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:02.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:20:02.298 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:02.297+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:02.303 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.303 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.312 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:20:02.399 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:02.398+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:02.457 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.458 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.466 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:02.507 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:02.506+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:02.600 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:02.599+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:02.610 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.610 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:02.701 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:02.699+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:02.764 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.764 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:02.801 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:02.800+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:02.877 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.877 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:02.886 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:02.902 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:02.901+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:02.991 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:02.991 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:03.000 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:03.004 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:03.003+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:03.065 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone17 from 0x4dc2555 (overlap 0x3c50bb1) to 0x60812b 2024-05-08T03:20:03.104 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:03.103+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:03.111 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:03.111 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:03.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:03.170 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:03.207 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:03.206+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:03.275 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:03.276 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:03.285 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:03.326 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:03.325+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:03.347 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:03.348 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:03.367 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:03.367 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:03.387 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:03.387 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:03.405 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:03.405 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:03.410 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:03.408+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:03.421 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:03.421 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:03.425 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:03.425 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:03.431 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:03.441 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-05-08T03:20:03.446 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:20:03.447 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:20:03.473 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:03.473 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:03.497 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:03.497 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:03.510 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:03.509+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:03.577 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:03.577 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:03.577 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:03.577 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:03.577 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:03.587 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:03.631 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:03.630+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:03.651 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:03.669 INFO:teuthology.orchestra.run.smithi177.stdout:1901 write 0xc8589ff thru 0xc85ba61 (0x3063 bytes) 2024-05-08T03:20:03.677 INFO:teuthology.orchestra.run.smithi177.stdout:1903 writesame 0xa148f8e thru 0xa157be3 data_size 0xa9(0xec56 bytes) 2024-05-08T03:20:03.686 INFO:teuthology.orchestra.run.smithi177.stdout:1904 read 0x116a965 thru 0x1177d0b (0xd3a7 bytes) 2024-05-08T03:20:03.688 INFO:teuthology.orchestra.run.smithi177.stdout:1905 punch from 0x8da2584 to 0x8da7f57, (0x59d3 bytes) 2024-05-08T03:20:03.688 INFO:teuthology.orchestra.run.smithi177.stdout:1906 write 0xbab53c5 thru 0xbac3fad (0xebe9 bytes) 2024-05-08T03:20:03.698 INFO:teuthology.orchestra.run.smithi177.stdout:1907 write 0xc2b88af thru 0xc2c678b (0xdedd bytes) 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.506990676000001, 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:03.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.506981226000001, 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:03.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:03.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:03.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.506968489999998, 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.506957913999997, 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:03.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:03.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:03.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:03.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.506948526999999, 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.726 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.506939346999999, 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.730 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276114311999997, 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.734 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:03.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:03.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:03.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276082074999998, 2024-05-08T03:20:03.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.742 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276068745000003, 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:03.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.746 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276053771000001, 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:03.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:03.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:03.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276042461999999, 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.754 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276024792999998, 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.755 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:03.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:03.758 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.276009594999998, 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:03.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:03.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:03.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 35.275995500999997, 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.763 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:03.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.501041528000002, 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.767 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:03.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 26.578147726000001, 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.776 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 26.578134238000001, 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:03.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:03.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:03.780 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 26.578105257000001, 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:03.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 26.578078650999998, 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.785 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 26.145330390000002, 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:03.789 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:03.794 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:03.795 INFO:teuthology.orchestra.run.smithi177.stdout:1911 read 0xa254616 thru 0xa26212a (0xdb15 bytes) 2024-05-08T03:20:03.796 INFO:teuthology.orchestra.run.smithi177.stdout:1912 punch from 0xaa4faa2 to 0xaa5320d, (0x376b bytes) 2024-05-08T03:20:03.796 INFO:teuthology.orchestra.run.smithi177.stdout:1913 read 0x2585fc thru 0x268281 (0xfc86 bytes) 2024-05-08T03:20:03.796 INFO:teuthology.orchestra.run.smithi177.stdout:1914 read 0x35340d1 thru 0x353cc1f (0x8b4f bytes) 2024-05-08T03:20:03.797 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:03.798 INFO:teuthology.orchestra.run.smithi177.stdout:1917 read 0x9baf449 thru 0x9bbc45b (0xd013 bytes) 2024-05-08T03:20:03.798 INFO:teuthology.orchestra.run.smithi177.stdout:1918 write 0x7954f96 thru 0x7961d67 (0xcdd2 bytes) 2024-05-08T03:20:03.798 INFO:teuthology.orchestra.run.smithi177.stdout:1919 write 0x74ec4a6 thru 0x74f91a6 (0xcd01 bytes) 2024-05-08T03:20:03.798 INFO:teuthology.orchestra.run.smithi177.stdout:1920 write 0x1815451 thru 0x181d702 (0x82b2 bytes) 2024-05-08T03:20:03.798 INFO:teuthology.orchestra.run.smithi177.stdout:1921 read 0x79d949c thru 0x79e15c6 (0x812b bytes) 2024-05-08T03:20:03.798 INFO:teuthology.orchestra.run.smithi177.stdout:1922 trunc from 0xc85ba62 to 0x3e34993 2024-05-08T03:20:03.798 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:03.794+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:03.809 INFO:teuthology.orchestra.run.smithi177.stdout:1923 read 0x31c6669 thru 0x31c8ac2 (0x245a bytes) 2024-05-08T03:20:03.811 INFO:teuthology.orchestra.run.smithi177.stdout:1924 read 0x1ef7b6 thru 0x1f07e9 (0x1034 bytes) 2024-05-08T03:20:03.815 INFO:teuthology.orchestra.run.smithi177.stdout:1925 writesame 0x449ded7 thru 0x449f83f data_size 0x1(0x1969 bytes) 2024-05-08T03:20:03.816 INFO:teuthology.orchestra.run.smithi177.stdout:1926 read 0xeafd39 thru 0xeb02e3 (0x5ab bytes) 2024-05-08T03:20:03.816 INFO:teuthology.orchestra.run.smithi177.stdout:1928 punch from 0x474246 to 0x47c835, (0x85ef bytes) 2024-05-08T03:20:03.821 INFO:teuthology.orchestra.run.smithi177.stdout:1929 trunc from 0x449f840 to 0x6b68c4e 2024-05-08T03:20:03.825 INFO:teuthology.orchestra.run.smithi177.stdout:1931 trunc from 0x6b68c4e to 0x761cb77 2024-05-08T03:20:03.843 INFO:teuthology.orchestra.run.smithi177.stdout:1932 write 0xe0716f3 thru 0xe07844c (0x6d5a bytes) 2024-05-08T03:20:03.853 INFO:teuthology.orchestra.run.smithi177.stdout:1934 writesame 0x799b6da thru 0x799da12 data_size 0x1(0x2339 bytes) 2024-05-08T03:20:03.859 INFO:teuthology.orchestra.run.smithi177.stdout:1935 read 0x1cd051c thru 0x1cdc812 (0xc2f7 bytes) 2024-05-08T03:20:03.861 INFO:teuthology.orchestra.run.smithi177.stdout:1936 punch from 0x1f1ee19 to 0x1f29c43, (0xae2a bytes) 2024-05-08T03:20:03.862 INFO:teuthology.orchestra.run.smithi177.stdout:1937 read 0x73a65c2 thru 0x73b2293 (0xbcd2 bytes) 2024-05-08T03:20:03.875 INFO:teuthology.orchestra.run.smithi177.stdout:1939 trunc from 0xe07844d to 0x24f4c30 2024-05-08T03:20:03.896 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:03.895+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:03.947 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:03.947 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:03.947 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:03.947 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:03.958 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:03.972 INFO:teuthology.orchestra.run.smithi177.stdout:1940 writesame 0xda7d6ea thru 0xda8692b data_size 0x2(0x9242 bytes) 2024-05-08T03:20:03.972 INFO:teuthology.orchestra.run.smithi177.stdout:1941 punch from 0x401925e to 0x4022550, (0x92f2 bytes) 2024-05-08T03:20:03.972 INFO:teuthology.orchestra.run.smithi177.stdout:1942 punch from 0x731b6b8 to 0x732842f, (0xcd77 bytes) 2024-05-08T03:20:03.972 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:03.972 INFO:teuthology.orchestra.run.smithi177.stdout:1944 punch from 0x469385d to 0x469e013, (0xa7b6 bytes) 2024-05-08T03:20:03.972 INFO:teuthology.orchestra.run.smithi177.stdout:1945 write 0x9cfdb75 thru 0x9d03795 (0x5c21 bytes) 2024-05-08T03:20:03.985 INFO:teuthology.orchestra.run.smithi177.stdout:1946 punch from 0xb5272bf to 0xb5315c0, (0xa301 bytes) 2024-05-08T03:20:03.985 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:03.985 INFO:teuthology.orchestra.run.smithi177.stdout:1948 punch from 0x63cc2de to 0x63d4d2e, (0x8a50 bytes) 2024-05-08T03:20:03.985 INFO:teuthology.orchestra.run.smithi177.stdout:1949 punch from 0x5b7bac to 0x5bafc1, (0x3415 bytes) 2024-05-08T03:20:03.985 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:03.985 INFO:teuthology.orchestra.run.smithi177.stdout:1951 punch from 0xc36dc11 to 0xc372016, (0x4405 bytes) 2024-05-08T03:20:03.997 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:03.997+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:04.001 INFO:teuthology.orchestra.run.smithi177.stdout:1952 trunc from 0xda8692c to 0x202625 2024-05-08T03:20:04.031 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:20:04.094 INFO:teuthology.orchestra.run.smithi177.stdout:1954 read 0x1931f thru 0x1a719 (0x13fb bytes) 2024-05-08T03:20:04.095 INFO:teuthology.orchestra.run.smithi177.stdout:1955 punch from 0x174cd2 to 0x17a404, (0x5732 bytes) 2024-05-08T03:20:04.098 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:04.097+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:04.101 INFO:teuthology.orchestra.run.smithi177.stdout:1956 write 0xedd218 thru 0xee15ea (0x43d3 bytes) 2024-05-08T03:20:04.108 INFO:teuthology.orchestra.run.smithi177.stdout:1957 read 0xdd1a7d thru 0xdd2e7b (0x13ff bytes) 2024-05-08T03:20:04.109 INFO:teuthology.orchestra.run.smithi177.stdout:1958 punch from 0xd0e35 to 0xdb0ab, (0xa276 bytes) 2024-05-08T03:20:04.110 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:04.111 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:04.111 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:04.111 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:04.111 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:04.119 INFO:teuthology.orchestra.run.smithi177.stdout:1961 write 0x57745b6 thru 0x5783b32 (0xf57d bytes) 2024-05-08T03:20:04.124 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:04.128 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.128 INFO:teuthology.orchestra.run.smithi177.stdout:1963 writesame 0x22ad851 thru 0x22ae8e4 data_size 0x2(0x1094 bytes) 2024-05-08T03:20:04.132 INFO:teuthology.orchestra.run.smithi177.stdout:1964 read 0x26ab3ad thru 0x26b81b3 (0xce07 bytes) 2024-05-08T03:20:04.133 INFO:teuthology.orchestra.run.smithi177.stdout:1965 read 0x30a982e thru 0x30aaf38 (0x170b bytes) 2024-05-08T03:20:04.133 INFO:teuthology.orchestra.run.smithi177.stdout:1966 read 0x45bb1d3 thru 0x45c907a (0xdea8 bytes) 2024-05-08T03:20:04.133 INFO:teuthology.orchestra.run.smithi177.stdout:1967 read 0x19ea483 thru 0x19f21f7 (0x7d75 bytes) 2024-05-08T03:20:04.133 INFO:teuthology.orchestra.run.smithi177.stdout:1968 read 0x1544318 thru 0x1549077 (0x4d60 bytes) 2024-05-08T03:20:04.144 INFO:teuthology.orchestra.run.smithi177.stdout:1969 writesame 0x9c2ff36 thru 0x9c3f811 data_size 0x6(0xf8dc bytes) 2024-05-08T03:20:04.144 INFO:teuthology.orchestra.run.smithi177.stdout:1971 read 0x277c6fd thru 0x2782f29 (0x682d bytes) 2024-05-08T03:20:04.144 INFO:teuthology.orchestra.run.smithi177.stdout:1972 read 0x7f6dd9 thru 0x8054f6 (0xe71e bytes) 2024-05-08T03:20:04.144 INFO:teuthology.orchestra.run.smithi177.stdout:1973 punch from 0x5206105 to 0x520da84, (0x797f bytes) 2024-05-08T03:20:04.144 INFO:teuthology.orchestra.run.smithi177.stdout:1975 read 0x3ffc88b thru 0x4003120 (0x6896 bytes) 2024-05-08T03:20:04.145 INFO:teuthology.orchestra.run.smithi177.stdout:1976 trunc from 0x9c3f812 to 0x933c466 2024-05-08T03:20:04.156 INFO:teuthology.orchestra.run.smithi177.stdout:1978 trunc from 0x933c466 to 0xdaa6a11 2024-05-08T03:20:04.162 INFO:teuthology.orchestra.run.smithi177.stdout:1980 write 0xa9b6c4e thru 0xa9bb73b (0x4aee bytes) 2024-05-08T03:20:04.173 INFO:teuthology.orchestra.run.smithi177.stdout:1982 flatten 2024-05-08T03:20:04.199 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:04.197+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:04.253 INFO:teuthology.orchestra.run.smithi177.stdout:1984 write 0xa8e0153 thru 0xa8e905c (0x8f0a bytes) 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:04.264 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.476395879999998, 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:04.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:04.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:04.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.329161730999999, 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:04.271 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:04.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:04.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:04.278 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.1390215640000001, 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:04.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:04.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:04.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.1142810179999998, 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:04.284 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:04.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.289 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25865 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [sparse-read 0~19262493] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.79241381799999999, 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053445962, 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.290 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25865 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469279+0000", 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.852e-06 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469291+0000", 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2098e-05 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469313+0000", 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2657999999999999e-05 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469326+0000", 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2782000000000001e-05 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469401+0000", 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4759999999999996e-05 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469495+0000", 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4000999999999997e-05 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.522722+0000", 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053226810999999999 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25866 3.1a 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.79179257800000002, 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049237975000000003, 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.294 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25866 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469898+0000", 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.23e-07 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469902+0000", 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4839999999999998e-06 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469913+0000", 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0652999999999999e-05 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469926+0000", 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3010000000000001e-05 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470021+0000", 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5622e-05 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470105+0000", 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3938999999999997e-05 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.519135+0000", 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049029544000000001 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.298 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25871 3.16 3:692ca2ad:::rbd_data.118227bb2f9.0000000000000006:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.79160580899999999, 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.047103610999999997, 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25871 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.06e-07 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470092+0000", 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8310000000000001e-06 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470099+0000", 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0999999999999998e-06 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470108+0000", 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3190000000000001e-06 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470162+0000", 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3618000000000001e-05 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470234+0000", 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2342999999999995e-05 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.517187+0000", 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046952894000000002 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.302 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27760 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 557056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.082468810000000004, 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046225914999999999, 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27760 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.303 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:04.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8900000000000001e-07 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179222+0000", 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.114e-06 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179225+0000", 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9189999999999999e-06 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179228+0000", 2024-05-08T03:20:04.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.819e-06 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225336+0000", 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046108300999999997 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225388+0000", 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2057e-05 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225447+0000", 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8415999999999998e-05 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27761 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.082458910999999996, 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046341465999999998, 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.307 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27761 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179232+0000", 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179235+0000", 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8770000000000002e-06 2024-05-08T03:20:04.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179237+0000", 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3690000000000001e-06 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225475+0000", 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046237568 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225515+0000", 2024-05-08T03:20:04.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0080000000000003e-05 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225572+0000", 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7351e-05 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27762 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 581632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.082449839999999996, 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046463320000000002, 2024-05-08T03:20:04.311 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27762 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.29e-07 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179241+0000", 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.263e-06 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179244+0000", 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7769999999999998e-06 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179246+0000", 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5509999999999998e-06 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225609+0000", 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046362154000000003 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225650+0000", 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225703+0000", 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3260999999999997e-05 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27763 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.082440262, 2024-05-08T03:20:04.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046573208999999997, 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27763 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179251+0000", 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.102e-06 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179253+0000", 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8229999999999999e-06 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179256+0000", 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4949999999999998e-06 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225727+0000", 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046471512999999999 2024-05-08T03:20:04.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225768+0000", 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0142999999999998e-05 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225822+0000", 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4926000000000002e-05 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.319 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27764 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.082430924000000003, 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046778976999999999, 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27764 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179381+0000", 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000122717 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179384+0000", 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5000000000000002e-06 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179389+0000", 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179392+0000", 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9280000000000003e-06 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225880+0000", 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046487354000000002 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225944+0000", 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4657000000000001e-05 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226038+0000", 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3066999999999995e-05 2024-05-08T03:20:04.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27765 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 802816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.082289383999999993, 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046843500000000003, 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27765 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181093+0000", 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016927820000000001 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181096+0000", 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.506e-06 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181103+0000", 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181108+0000", 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5979999999999996e-06 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226080+0000", 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044971481000000001 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226152+0000", 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2057000000000005e-05 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226244+0000", 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1891000000000005e-05 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27776 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.080352844000000007, 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046303351999999999, 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.328 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27776 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2599999999999998e-07 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181338+0000", 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.266e-06 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181341+0000", 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181344+0000", 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227544+0000", 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046199656999999998 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227585+0000", 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1850000000000001e-05 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227640+0000", 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4586000000000002e-05 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27777 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.332 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.080342649000000002, 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046409785000000002, 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27777 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:04.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181348+0000", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.145e-06 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181351+0000", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8210000000000002e-06 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181354+0000", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5900000000000002e-06 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227662+0000", 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046308716 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227703+0000", 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0141000000000001e-05 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227757+0000", 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4147000000000002e-05 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27778 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.080332927999999998, 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046521521000000003, 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27778 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.337 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:04.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181358+0000", 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.167e-06 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181362+0000", 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181364+0000", 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6019999999999998e-06 2024-05-08T03:20:04.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227780+0000", 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046415911999999997 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227821+0000", 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1179e-05 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227878+0000", 2024-05-08T03:20:04.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6736e-05 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27779 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.080322394000000005, 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046626723000000002, 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27779 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183095+0000", 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001728282 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183102+0000", 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4790000000000001e-06 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183117+0000", 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4929999999999999e-05 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183126+0000", 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9209999999999998e-06 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227901+0000", 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044775342000000003 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227940+0000", 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9332999999999998e-05 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227994+0000", 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3436000000000001e-05 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27794 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1269760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.078183692999999999, 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046263348000000003, 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.345 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27794 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183508+0000", 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.057e-06 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183514+0000", 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6710000000000004e-06 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183519+0000", 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8269999999999997e-06 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229676+0000", 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046157090999999997 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229716+0000", 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9870000000000003e-05 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229769+0000", 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3477000000000003e-05 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27795 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1294336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.349 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.078164829000000005, 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046360547000000002, 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27795 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:20:04.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183527+0000", 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183533+0000", 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183537+0000", 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4660000000000001e-06 2024-05-08T03:20:04.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229792+0000", 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046254587999999999 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229832+0000", 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0055999999999997e-05 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229885+0000", 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3488e-05 2024-05-08T03:20:04.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27796 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1441792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.078146381000000001, 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046459223000000001, 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.354 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27796 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185095+0000", 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015521389999999999 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185101+0000", 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185115+0000", 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3735e-05 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185123+0000", 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1359999999999997e-06 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229910+0000", 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044786908 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229949+0000", 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9192e-05 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.230002+0000", 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3601999999999999e-05 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.358 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:04.359 INFO:teuthology.orchestra.run.smithi177.stdout:1985 read 0x81789ca thru 0x8182d30 (0xa367 bytes) 2024-05-08T03:20:04.359 INFO:teuthology.orchestra.run.smithi177.stdout:1988 read 0x36a405 thru 0x36a56e (0x16a bytes) 2024-05-08T03:20:04.360 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:04.361 INFO:teuthology.orchestra.run.smithi177.stdout:1990 trunc from 0xdaa6a11 to 0x17df38c 2024-05-08T03:20:04.361 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.361 INFO:teuthology.orchestra.run.smithi177.stdout:1992 read 0xb6f614 thru 0xb7693c (0x7329 bytes) 2024-05-08T03:20:04.361 INFO:teuthology.orchestra.run.smithi177.stdout:1994 read 0xa29052 thru 0xa2d934 (0x48e3 bytes) 2024-05-08T03:20:04.361 INFO:teuthology.orchestra.run.smithi177.stdout:1995 writesame 0xf677c6 thru 0xf73845 data_size 0xb0(0xc080 bytes) 2024-05-08T03:20:04.361 INFO:teuthology.orchestra.run.smithi177.stdout:1996 writesame 0xe35e9c9 thru 0xe35feda data_size 0x1f(0x1512 bytes) 2024-05-08T03:20:04.362 INFO:teuthology.orchestra.run.smithi177.stdout:1997 write 0x7c2185e thru 0x7c26731 (0x4ed4 bytes) 2024-05-08T03:20:04.362 INFO:teuthology.orchestra.run.smithi177.stdout:1999 write 0x3779340 thru 0x3787bf6 (0xe8b7 bytes) 2024-05-08T03:20:04.362 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:04.358+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:04.367 INFO:teuthology.orchestra.run.smithi177.stdout:2001 write 0x30ad9cc thru 0x30b8c90 (0xb2c5 bytes) 2024-05-08T03:20:04.377 INFO:teuthology.orchestra.run.smithi177.stdout:2004 read 0x162b248 thru 0x1639740 (0xe4f9 bytes) 2024-05-08T03:20:04.377 INFO:teuthology.orchestra.run.smithi177.stdout:2008 read 0x2c3fa7e thru 0x2c43d4e (0x42d1 bytes) 2024-05-08T03:20:04.385 INFO:teuthology.orchestra.run.smithi177.stdout:2011 trunc from 0xe35fedb to 0x74eb55e 2024-05-08T03:20:04.409 INFO:teuthology.orchestra.run.smithi177.stdout:2012 write 0xd1c38a0 thru 0xd1c7b30 (0x4291 bytes) 2024-05-08T03:20:04.418 INFO:teuthology.orchestra.run.smithi177.stdout:2014 punch from 0x6cd2d55 to 0x6cdf786, (0xca31 bytes) 2024-05-08T03:20:04.418 INFO:teuthology.orchestra.run.smithi177.stdout:2015 read 0x31902e3 thru 0x3199d82 (0x9aa0 bytes) 2024-05-08T03:20:04.418 INFO:teuthology.orchestra.run.smithi177.stdout:2016 read 0x97b279b thru 0x97b67fd (0x4063 bytes) 2024-05-08T03:20:04.418 INFO:teuthology.orchestra.run.smithi177.stdout:2017 read 0x54aa0b1 thru 0x54b64d7 (0xc427 bytes) 2024-05-08T03:20:04.418 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.418 INFO:teuthology.orchestra.run.smithi177.stdout:2019 writesame 0x46dc153 thru 0x46ea7c6 data_size 0x2b(0xe674 bytes) 2024-05-08T03:20:04.427 INFO:teuthology.orchestra.run.smithi177.stdout:2020 write 0x6cfde7b thru 0x6d03f65 (0x60eb bytes) 2024-05-08T03:20:04.437 INFO:teuthology.orchestra.run.smithi177.stdout:2021 write 0x4a3d871 thru 0x4a4c0ab (0xe83b bytes) 2024-05-08T03:20:04.449 INFO:teuthology.orchestra.run.smithi177.stdout:2024 write 0x44c6fd6 thru 0x44d2fb4 (0xbfdf bytes) 2024-05-08T03:20:04.457 INFO:teuthology.orchestra.run.smithi177.stdout:2025 punch from 0x43db7ff to 0x43e681e, (0xb01f bytes) 2024-05-08T03:20:04.457 INFO:teuthology.orchestra.run.smithi177.stdout:2026 punch from 0xb53bcc8 to 0xb547ee5, (0xc21d bytes) 2024-05-08T03:20:04.457 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.457 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.457 INFO:teuthology.orchestra.run.smithi177.stdout:2029 writesame 0xc9548ea thru 0xc95b4a6 data_size 0x1(0x6bbd bytes) 2024-05-08T03:20:04.459 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:04.458+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:04.469 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.469 INFO:teuthology.orchestra.run.smithi177.stdout:2033 trunc from 0xd1c7b31 to 0xd33d2e7 2024-05-08T03:20:04.474 INFO:teuthology.orchestra.run.smithi177.stdout:2034 read 0xca0c960 thru 0xca0e9b2 (0x2053 bytes) 2024-05-08T03:20:04.474 INFO:teuthology.orchestra.run.smithi177.stdout:2035 write 0x15a763c thru 0x15b1da0 (0xa765 bytes) 2024-05-08T03:20:04.484 INFO:teuthology.orchestra.run.smithi177.stdout:2036 write 0x4421003 thru 0x4429394 (0x8392 bytes) 2024-05-08T03:20:04.487 INFO:teuthology.orchestra.run.smithi177.stdout:2037 read 0x361e44a thru 0x361fe95 (0x1a4c bytes) 2024-05-08T03:20:04.491 INFO:teuthology.orchestra.run.smithi177.stdout:2038 trunc from 0xd33d2e7 to 0xa25469a 2024-05-08T03:20:04.501 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:04.501 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:04.501 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:04.501 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:04.502 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.502 INFO:teuthology.orchestra.run.smithi177.stdout:2041 punch from 0x63a75b3 to 0x63b6351, (0xed9e bytes) 2024-05-08T03:20:04.502 INFO:teuthology.orchestra.run.smithi177.stdout:2044 punch from 0x8cae1c1 to 0x8cae271, (0xb0 bytes) 2024-05-08T03:20:04.502 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.503 INFO:teuthology.orchestra.run.smithi177.stdout:2046 punch from 0x654821 to 0x65faf8, (0xb2d7 bytes) 2024-05-08T03:20:04.503 INFO:teuthology.orchestra.run.smithi177.stdout:2047 write 0x8d92a5d thru 0x8d9fd33 (0xd2d7 bytes) 2024-05-08T03:20:04.511 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:04.517 INFO:teuthology.orchestra.run.smithi177.stdout:2048 trunc from 0xa25469a to 0x73ae54c 2024-05-08T03:20:04.527 INFO:teuthology.orchestra.run.smithi177.stdout:2049 write 0x53f9162 thru 0x5402374 (0x9213 bytes) 2024-05-08T03:20:04.540 INFO:teuthology.orchestra.run.smithi177.stdout:2050 read 0x4f7a37b thru 0x4f7e9d6 (0x465c bytes) 2024-05-08T03:20:04.541 INFO:teuthology.orchestra.run.smithi177.stdout:2051 trunc from 0x73ae54c to 0x783fff7 2024-05-08T03:20:04.545 INFO:teuthology.orchestra.run.smithi177.stdout:2053 writesame 0x2e76792 thru 0x2e7f65b data_size 0x31(0x8eca bytes) 2024-05-08T03:20:04.561 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:04.560+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:04.568 INFO:teuthology.orchestra.run.smithi177.stdout:2054 write 0xc0890eb thru 0xc092203 (0x9119 bytes) 2024-05-08T03:20:04.579 INFO:teuthology.orchestra.run.smithi177.stdout:2055 read 0x6ae1b2e thru 0x6aec499 (0xa96c bytes) 2024-05-08T03:20:04.579 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.579 INFO:teuthology.orchestra.run.smithi177.stdout:2059 punch from 0x91d182f to 0x91dc9ce, (0xb19f bytes) 2024-05-08T03:20:04.579 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.579 INFO:teuthology.orchestra.run.smithi177.stdout:2061 punch from 0xa5ae806 to 0xa5afbda, (0x13d4 bytes) 2024-05-08T03:20:04.579 INFO:teuthology.orchestra.run.smithi177.stdout:2062 write 0x7808897 thru 0x7814971 (0xc0db bytes) 2024-05-08T03:20:04.592 INFO:teuthology.orchestra.run.smithi177.stdout:2063 write 0xa431d4e thru 0xa441b7a (0xfe2d bytes) 2024-05-08T03:20:04.603 INFO:teuthology.orchestra.run.smithi177.stdout:2064 writesame 0x62baba2 thru 0x62ca89c data_size 0x1(0xfcfb bytes) 2024-05-08T03:20:04.622 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:04.622 INFO:teuthology.orchestra.run.smithi177.stdout:2066 clone 19 order 18 su 4096 sc 7 2024-05-08T03:20:04.644 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:04.645 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:04.653 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:04.662 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:04.661+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:04.764 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:04.763+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12442 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:04.801 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 28.594436225999999, 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036923198999999997, 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12442 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204568+0000", 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6500000000000001e-06 2024-05-08T03:20:04.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207214+0000", 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0026454019999999998 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207240+0000", 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6523e-05 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207254+0000", 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.43e-05 2024-05-08T03:20:04.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207335+0000", 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0395000000000005e-05 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207518+0000", 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182819 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208440+0000", 2024-05-08T03:20:04.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00092235600000000004 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225462+0000", 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017022065999999999 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241319+0000", 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015856677 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241446+0000", 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012764100000000001 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241490+0000", 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.337e-05 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12822 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.807 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671937023000002, 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089581330000000001, 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12822 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127066+0000", 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5499999999999998e-07 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127069+0000", 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2740000000000002e-06 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127084+0000", 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5495999999999999e-05 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127093+0000", 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2779999999999999e-06 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135432+0000", 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083386629999999996 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.135511+0000", 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9437000000000005e-05 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136024+0000", 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051252999999999997 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12835 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 524288~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671518634000002, 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0092348480000000004, 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12835 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127484+0000", 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127485+0000", 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0500000000000002e-07 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127490+0000", 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1669999999999998e-06 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127500+0000", 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0257e-05 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127507+0000", 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4760000000000003e-06 2024-05-08T03:20:04.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136070+0000", 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.008563279 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136160+0000", 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9445999999999997e-05 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136719+0000", 2024-05-08T03:20:04.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000559418 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12836 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2621440~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671467356000001, 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097351599999999996, 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12836 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127535+0000", 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127536+0000", 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2399999999999998e-07 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127539+0000", 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1269999999999999e-06 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127547+0000", 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4529999999999996e-06 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127552+0000", 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4700000000000001e-06 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136745+0000", 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0091925099999999992 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.136801+0000", 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6663999999999998e-05 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137271+0000", 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000469412 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12851 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 655360~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671442267, 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010145049999999999, 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12851 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127561+0000", 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6199999999999998e-07 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127563+0000", 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4140000000000001e-06 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127569+0000", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127574+0000", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137312+0000", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097380450000000007 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137391+0000", 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9394000000000006e-05 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137706+0000", 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00031446099999999998 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.824 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12852 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2752512~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671423325999999, 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010659394000000001, 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12852 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:20:04.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127580+0000", 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9000000000000002e-07 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127582+0000", 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127589+0000", 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5919999999999997e-06 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127592+0000", 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8360000000000002e-06 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137740+0000", 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010148121 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.137820+0000", 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9524999999999995e-05 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138239+0000", 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00041890800000000001 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12867 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671405725, 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011192809999999999, 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12867 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.829 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127597+0000", 2024-05-08T03:20:04.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127599+0000", 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.894e-06 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127605+0000", 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127610+0000", 2024-05-08T03:20:04.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8080000000000004e-06 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138262+0000", 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.010652069 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138333+0000", 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0634000000000002e-05 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138790+0000", 2024-05-08T03:20:04.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00045714900000000001 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12868 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 2883584~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671386627, 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011717782, 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.833 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12868 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127616+0000", 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127617+0000", 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.53e-07 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127619+0000", 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4870000000000001e-06 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127625+0000", 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3029999999999996e-06 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127631+0000", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9900000000000002e-06 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138833+0000", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011201871 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.138902+0000", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9194000000000002e-05 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139334+0000", 2024-05-08T03:20:04.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00043158399999999999 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12883 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671365056999999, 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012139397, 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.837 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12883 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127638+0000", 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127641+0000", 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127648+0000", 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7989999999999996e-06 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127653+0000", 2024-05-08T03:20:04.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:04.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139365+0000", 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01171205 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139435+0000", 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0135000000000005e-05 2024-05-08T03:20:04.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139777+0000", 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00034227200000000002 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12884 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3014656~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.671343688, 2024-05-08T03:20:04.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012668127, 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12884 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127659+0000", 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127660+0000", 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127661+0000", 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.962e-06 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127668+0000", 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.105e-06 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.127673+0000", 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1649999999999997e-06 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139800+0000", 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012127712000000001 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.139884+0000", 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3873000000000001e-05 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140327+0000", 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000442939 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12899 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1048576~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.846 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670310488999998, 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012150764999999999, 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12899 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128692+0000", 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128694+0000", 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1739999999999999e-06 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128699+0000", 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1540000000000001e-06 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128714+0000", 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5755000000000001e-05 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128722+0000", 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9789999999999993e-06 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140360+0000", 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011637522000000001 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140439+0000", 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9297999999999999e-05 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140843+0000", 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00040388300000000001 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.850 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12900 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3145728~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670266255000001, 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012610682999999999, 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12900 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128737+0000", 2024-05-08T03:20:04.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.44e-07 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128740+0000", 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2949999999999998e-06 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128748+0000", 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.43e-06 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128753+0000", 2024-05-08T03:20:04.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3399999999999997e-06 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140875+0000", 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012122198000000001 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.140951+0000", 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6036000000000002e-05 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141347+0000", 2024-05-08T03:20:04.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00039584 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12915 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1179648~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670242185999999, 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013175711999999999, 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.855 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12915 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128761+0000", 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7100000000000002e-07 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128765+0000", 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5300000000000001e-06 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128772+0000", 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7589999999999997e-06 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128778+0000", 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1000000000000003e-06 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141378+0000", 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.012600668000000001 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141432+0000", 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3448000000000001e-05 2024-05-08T03:20:04.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141936+0000", 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00050473599999999999 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12916 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670217176000001, 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013781595000000001, 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.859 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12916 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:20:04.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128786+0000", 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5900000000000002e-07 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128788+0000", 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2010000000000002e-06 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128795+0000", 2024-05-08T03:20:04.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6730000000000003e-06 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128800+0000", 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3550000000000004e-06 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.141966+0000", 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013165469000000001 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142011+0000", 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5331999999999999e-05 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142567+0000", 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000556106 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12931 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1310720~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670195558, 2024-05-08T03:20:04.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014420924, 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12931 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128807+0000", 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128808+0000", 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.51e-07 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128810+0000", 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.238e-06 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128816+0000", 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128821+0000", 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2499999999999997e-06 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142597+0000", 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.013775832999999999 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.142679+0000", 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1757999999999999e-05 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143228+0000", 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054913200000000003 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12932 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:20:04.867 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670174982999999, 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015052658, 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12932 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128828+0000", 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9900000000000001e-07 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128830+0000", 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0999999999999998e-06 2024-05-08T03:20:04.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128837+0000", 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3080000000000004e-06 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128842+0000", 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8910000000000003e-06 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143270+0000", 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.014428778999999999 2024-05-08T03:20:04.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143335+0000", 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5162999999999999e-05 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143881+0000", 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054501799999999998 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12947 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 1441792~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670154915000001, 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015679150999999999, 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12947 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128848+0000", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128850+0000", 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.013e-06 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128856+0000", 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0630000000000002e-06 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128861+0000", 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9540000000000003e-06 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.143935+0000", 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015074086 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144017+0000", 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1118000000000005e-05 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144527+0000", 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051051099999999995 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12948 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:a [sparse-read 3538944~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.670134694000001, 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016271184000000001, 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12948 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128868+0000", 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128869+0000", 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:20:04.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128871+0000", 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128877+0000", 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128882+0000", 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2719999999999997e-06 2024-05-08T03:20:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144554+0000", 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015671378999999999 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.144626+0000", 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2265e-05 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145139+0000", 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00051343000000000005 2024-05-08T03:20:04.879 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:21013 3.9 3:90c815cd:::rbd_data.1182cef43756.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:51.075985+0000", 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.723017535, 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.011481404000000001, 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:04.880 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 21013 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.075985+0000", 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.075985+0000", 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.075987+0000", 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5740000000000001e-06 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.075994+0000", 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8170000000000001e-06 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.076008+0000", 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4643e-05 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.076016+0000", 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8900000000000007e-06 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.076074+0000", 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7824000000000001e-05 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.076161+0000", 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7311000000000005e-05 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:04.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.077874+0000", 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0017128079999999999 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.087372+0000", 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0094977120000000002 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.087386+0000", 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3957999999999999e-05 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.087425+0000", 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9323999999999998e-05 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:51.087467+0000", 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1542999999999998e-05 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23265 3.1a e55/28 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head v 55'298, mlcod=55'295)", 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.086832+0000", 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.7121711419999999, 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020384744, 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086832+0000", 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086832+0000", 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086833+0000", 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.513e-06 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086844+0000", 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1090000000000001e-05 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086864+0000", 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0055999999999999e-05 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086875+0000", 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0651e-05 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086951+0000", 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6365000000000005e-05 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086974+0000", 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2289e-05 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.107152+0000", 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020178048000000001 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.107216+0000", 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4732000000000003e-05 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:04.890 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:04.890 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:04.891 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:04.892 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:04.890+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:04.992 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:04.991+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:05.025 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:20:05.044 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:05.044 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:05.045 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:05.045 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:05.054 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:05.093 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:05.092+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:05.194 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:05.193+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:05.198 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:05.199 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:05.207 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:05.295 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:05.294+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:05.314 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3155886040000002, 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.3155211399999995, 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:05.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.2556868080000001, 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.327 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1891787950000001, 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:05.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.338 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1626411489999997, 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:05.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:05.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:05.344 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1347293809999996, 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:05.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23281 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'92, mlcod=53'88)", 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.0851838620000001, 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074279843999999998, 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:05.349 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226785+0000", 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.299e-06 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226794+0000", 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5040000000000008e-06 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226810+0000", 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5781e-05 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226820+0000", 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9310000000000003e-06 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262249+0000", 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.035428501000000001 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262269+0000", 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0077000000000001e-05 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301028+0000", 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.038759184000000002 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301064+0000", 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5567e-05 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26865 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4288512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:05.353 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.417570996, 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041924771999999999, 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26865 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:05.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894399+0000", 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6309999999999999e-06 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894403+0000", 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8559999999999998e-06 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894408+0000", 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9259999999999999e-06 2024-05-08T03:20:05.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936202+0000", 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041793779000000003 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936255+0000", 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3072000000000003e-05 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936321+0000", 2024-05-08T03:20:05.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6128000000000001e-05 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26866 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4460544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.417554559, 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042116690999999998, 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26866 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0400000000000002e-07 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894415+0000", 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5990000000000001e-06 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894420+0000", 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9119999999999997e-06 2024-05-08T03:20:05.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894424+0000", 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7299999999999999e-06 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936364+0000", 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041939999999999998 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936447+0000", 2024-05-08T03:20:05.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3293000000000004e-05 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936530+0000", 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2853000000000001e-05 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26867 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4509696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4175397620000001, 2024-05-08T03:20:05.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042262936000000001, 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26867 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896093+0000", 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016655789999999999 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896098+0000", 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8180000000000003e-06 2024-05-08T03:20:05.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896109+0000", 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1109e-05 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896118+0000", 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8489999999999995e-06 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936561+0000", 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040442845999999998 2024-05-08T03:20:05.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936610+0000", 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9286000000000002e-05 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936691+0000", 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0449000000000002e-05 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26874 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4386816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.415712914, 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041925208999999998, 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26874 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:05.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896258+0000", 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9979999999999999e-06 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896261+0000", 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1470000000000001e-06 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896265+0000", 2024-05-08T03:20:05.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9419999999999997e-06 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938052+0000", 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041787283000000001 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938114+0000", 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2020000000000006e-05 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938180+0000", 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5611999999999995e-05 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26875 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4411392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4156975520000001, 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.04205014, 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26875 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9200000000000002e-07 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896272+0000", 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6789999999999999e-06 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896277+0000", 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8289999999999998e-06 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896281+0000", 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.839e-06 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938214+0000", 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041933639000000002 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938260+0000", 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5510000000000003e-05 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938320+0000", 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0251999999999998e-05 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26876 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4435968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:05.375 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.415681032, 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042168543000000003, 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26876 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896289+0000", 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9470000000000002e-06 2024-05-08T03:20:05.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896292+0000", 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5470000000000001e-06 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896296+0000", 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.574e-06 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938354+0000", 2024-05-08T03:20:05.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042057861000000002 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938398+0000", 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4221999999999998e-05 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938455+0000", 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7037999999999998e-05 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26877 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4608000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4156681010000001, 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042279939000000002, 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26877 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.380 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896300+0000", 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:05.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896301+0000", 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.159e-06 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896304+0000", 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.881e-06 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896307+0000", 2024-05-08T03:20:05.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938478+0000", 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042171023000000002 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938524+0000", 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6109999999999997e-05 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938579+0000", 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5448000000000002e-05 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26878 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4632576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4156575920000001, 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042399394, 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26878 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898092+0000", 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0017816349999999999 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898096+0000", 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7720000000000002e-06 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898110+0000", 2024-05-08T03:20:05.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3764000000000001e-05 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898118+0000", 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1659999999999994e-06 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938611+0000", 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040492570999999998 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938654+0000", 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3133000000000002e-05 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938709+0000", 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5352999999999997e-05 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26892 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4853760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.388 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4135112300000001, 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041959164, 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26892 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898457+0000", 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3399999999999999e-07 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898459+0000", 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2960000000000001e-06 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898466+0000", 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2400000000000001e-06 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898471+0000", 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.532e-06 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940319+0000", 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041847652999999999 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940360+0000", 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0952999999999999e-05 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940415+0000", 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5955999999999998e-05 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.392 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26893 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4878336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4134907809999999, 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042048864999999998, 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.393 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26893 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898479+0000", 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898489+0000", 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6900000000000004e-06 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898493+0000", 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5709999999999999e-06 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940438+0000", 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041944253000000001 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940475+0000", 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7512000000000003e-05 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940526+0000", 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0599000000000001e-05 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26894 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4134657159999999, 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042154042000000003, 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.397 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26894 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900091+0000", 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015893400000000001 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900096+0000", 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900110+0000", 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3773e-05 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900119+0000", 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3880000000000006e-06 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940559+0000", 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040439875 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940601+0000", 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1950999999999999e-05 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940656+0000", 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4960999999999998e-05 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.401 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26909 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.4115023250000001, 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041961032000000002, 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26909 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900466+0000", 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3799999999999998e-07 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900468+0000", 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.447e-06 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900475+0000", 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9380000000000003e-06 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900479+0000", 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2030000000000002e-06 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942337+0000", 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041857710999999999 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942376+0000", 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9533999999999998e-05 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942426+0000", 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9861000000000003e-05 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.405 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26910 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 5566464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.411482326, 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042062763000000003, 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:05.406 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26910 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902091+0000", 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001605778 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902096+0000", 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5759999999999999e-06 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902108+0000", 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2018000000000001e-05 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902116+0000", 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8429999999999999e-06 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942456+0000", 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040339331999999999 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942496+0000", 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9774999999999998e-05 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:05.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942548+0000", 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2441000000000002e-05 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:05.410 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:05.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:05.412 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:05.410+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:05.513 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:05.511+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:05.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:05.527 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:05.527 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:05.528 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:05.528 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:05.537 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:05.614 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:05.613+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:05.635 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:05.636 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:05.642 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:05.642 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:05.643 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:05.643 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:05.643 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:05.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:05.653 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:05.653 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:05.669 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:05.669 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:05.686 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:05.686 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:05.703 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:05.703 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:05.715 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:05.714+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:05.720 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:20:05.720 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:20:05.737 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:05.737 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:05.754 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:05.754 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:05.797 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:05.797 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:05.805 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:20:05.816 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:05.814+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:05.910 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:05.910 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:05.917 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:05.915+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:05.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:20:06.017 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:06.016+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:06.025 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.025 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.035 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:20:06.118 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:06.117+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:06.138 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.139 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.147 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:06.219 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:06.217+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:06.293 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.293 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:06.320 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:06.318+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:06.407 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.407 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.415 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:06.421 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:06.419+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:06.520 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.520 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.521 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:06.520+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:06.529 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:06.622 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:06.620+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:06.675 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.675 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.683 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:06.723 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:06.722+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:06.789 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.789 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.797 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:06.841 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:06.840+0000 7fe7fcbfb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32749) UID: 0 2024-05-08T03:20:06.902 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:06.902 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:06.911 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:06.955 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:06.954+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:07.026 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:07.025+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:07.030 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6] out_osds: [4, 7, 0, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:20:07.031 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:07.031 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T03:20:07.031 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T03:20:07.031 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:20:07.031 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:07.031 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:07.043 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:20:07.045 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:07.058 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T03:20:07.059 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-05-08T03:20:07.059 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:20:07.059 DEBUG:teuthology.orchestra.run.smithi188:> type ceph-objectstore-tool 2024-05-08T03:20:07.113 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T03:20:07.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-05-08T03:20:07.192 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:07.129+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:07.211 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:07.211 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:07.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:07.265 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:07.264+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:07.313 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-05-08T03:20:07.328 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:07.327+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:07.334 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.335 INFO:teuthology.orchestra.run.smithi177.stdout:2068 read 0x8b382d5 thru 0x8b38563 (0x28f bytes) 2024-05-08T03:20:07.335 INFO:teuthology.orchestra.run.smithi177.stdout:2069 writesame 0x6ec9676 thru 0x6ecf2bb data_size 0x6(0x5c46 bytes) 2024-05-08T03:20:07.346 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:07.346 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:07.349 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.349 INFO:teuthology.orchestra.run.smithi177.stdout:2072 read 0xa0c4333 thru 0xa0c9aad (0x577b bytes) 2024-05-08T03:20:07.349 INFO:teuthology.orchestra.run.smithi177.stdout:2073 write 0x27eb0cc thru 0x27ed6d1 (0x2606 bytes) 2024-05-08T03:20:07.356 INFO:teuthology.orchestra.run.smithi177.stdout:2074 writesame 0x7beb014 thru 0x7bf3108 data_size 0x1(0x80f5 bytes) 2024-05-08T03:20:07.358 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:07.372 INFO:teuthology.orchestra.run.smithi177.stdout:2075 read 0x98b19f1 thru 0x98b50df (0x36ef bytes) 2024-05-08T03:20:07.373 INFO:teuthology.orchestra.run.smithi177.stdout:2077 write 0x6f4b1cc thru 0x6f56a4e (0xb883 bytes) 2024-05-08T03:20:07.378 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.378 INFO:teuthology.orchestra.run.smithi177.stdout:2081 punch from 0x178d36b to 0x1795808, (0x849d bytes) 2024-05-08T03:20:07.378 INFO:teuthology.orchestra.run.smithi177.stdout:2082 punch from 0x5d7ba44 to 0x5d7c3d7, (0x993 bytes) 2024-05-08T03:20:07.378 INFO:teuthology.orchestra.run.smithi177.stdout:2083 writesame 0x5ef86c9 thru 0x5f07fdf data_size 0x1f(0xf917 bytes) 2024-05-08T03:20:07.389 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.389 INFO:teuthology.orchestra.run.smithi177.stdout:2085 read 0x8a626e5 thru 0x8a6b8b0 (0x91cc bytes) 2024-05-08T03:20:07.389 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.389 INFO:teuthology.orchestra.run.smithi177.stdout:2088 read 0x3eed6c0 thru 0x3eefa8e (0x23cf bytes) 2024-05-08T03:20:07.389 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.389 INFO:teuthology.orchestra.run.smithi177.stdout:2093 write 0x646b868 thru 0x6476c40 (0xb3d9 bytes) 2024-05-08T03:20:07.401 INFO:teuthology.orchestra.run.smithi177.stdout:2094 read 0x7bf15a1 thru 0x7bf4124 (0x2b84 bytes) 2024-05-08T03:20:07.402 INFO:teuthology.orchestra.run.smithi177.stdout:2095 write 0x773a7e9 thru 0x7746e0e (0xc626 bytes) 2024-05-08T03:20:07.423 INFO:teuthology.orchestra.run.smithi177.stdout:2096 writesame 0x945b964 thru 0x945fb38 data_size 0x1(0x41d5 bytes) 2024-05-08T03:20:07.429 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:07.428+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:07.435 INFO:teuthology.orchestra.run.smithi177.stdout:2097 trunc from 0xc092204 to 0xabac8a5 2024-05-08T03:20:07.455 INFO:teuthology.orchestra.run.smithi177.stdout:2098 read 0xa9f2042 thru 0xa9f2fde (0xf9d bytes) 2024-05-08T03:20:07.455 INFO:teuthology.orchestra.run.smithi177.stdout:2099 read 0xaac0af1 thru 0xaacb0fd (0xa60d bytes) 2024-05-08T03:20:07.455 INFO:teuthology.orchestra.run.smithi177.stdout:2100 punch from 0x6587e3 to 0x65957b, (0xd98 bytes) 2024-05-08T03:20:07.455 INFO:teuthology.orchestra.run.smithi177.stdout:2101 write 0x6d4f4ac thru 0x6d55952 (0x64a7 bytes) 2024-05-08T03:20:07.488 INFO:teuthology.orchestra.run.smithi177.stdout:2102 read 0x4e3900f thru 0x4e3de91 (0x4e83 bytes) 2024-05-08T03:20:07.488 INFO:teuthology.orchestra.run.smithi177.stdout:2105 read 0x247f935 thru 0x248399f (0x406b bytes) 2024-05-08T03:20:07.488 INFO:teuthology.orchestra.run.smithi177.stdout:2107 read 0x51a9927 thru 0x51b125d (0x7937 bytes) 2024-05-08T03:20:07.488 INFO:teuthology.orchestra.run.smithi177.stdout:2108 write 0x37ad0c4 thru 0x37b71d9 (0xa116 bytes) 2024-05-08T03:20:07.523 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:07.524 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:07.525 INFO:teuthology.orchestra.run.smithi177.stdout:2109 punch from 0x9988862 to 0x9997239, (0xe9d7 bytes) 2024-05-08T03:20:07.530 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:07.529+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:07.531 INFO:teuthology.orchestra.run.smithi177.stdout:2112 write 0xb070246 thru 0xb07aa54 (0xa80f bytes) 2024-05-08T03:20:07.534 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:07.545 INFO:teuthology.orchestra.run.smithi177.stdout:2113 read 0x63a83af thru 0x63b17cf (0x9421 bytes) 2024-05-08T03:20:07.546 INFO:teuthology.orchestra.run.smithi177.stdout:2114 write 0x6ad8282 thru 0x6adb609 (0x3388 bytes) 2024-05-08T03:20:07.562 INFO:teuthology.orchestra.run.smithi177.stdout:2115 trunc from 0xb07aa55 to 0x565fd28 2024-05-08T03:20:07.617 INFO:teuthology.orchestra.run.smithi177.stdout:2118 writesame 0x719e3fe thru 0x71a446b data_size 0x1(0x606e bytes) 2024-05-08T03:20:07.617 INFO:teuthology.orchestra.run.smithi177.stdout:2120 read 0x6eea864 thru 0x6eeb800 (0xf9d bytes) 2024-05-08T03:20:07.617 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:07.617 INFO:teuthology.orchestra.run.smithi177.stdout:2122 write 0x1526c1b thru 0x153295c (0xbd42 bytes) 2024-05-08T03:20:07.632 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:07.630+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:07.678 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:07.678 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:07.687 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:07.766 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:07.772 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:07.770+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:07.833 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:07.832+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:07.836 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:07.836 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:07.851 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:07.858 INFO:teuthology.orchestra.run.smithi177.stdout:2123 trunc from 0x71a446c to 0x4c40342 2024-05-08T03:20:07.881 INFO:teuthology.orchestra.run.smithi177.stdout:2124 clone 20 order 19 su 32768 sc 6 2024-05-08T03:20:07.906 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:07.907 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:07.924 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:07.924 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:07.934 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:07.933+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:07.941 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:07.941 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:07.958 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:07.958 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:07.969 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:07.969 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:07.969 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:07.969 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:07.976 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:07.976 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:07.978 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:08.035 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:08.034+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:08.043 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:20:08.099 INFO:teuthology.orchestra.run.smithi177.stdout:flattening image image_client.0-clone19 2024-05-08T03:20:08.136 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:08.135+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:08.158 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:08.158 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:08.159 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:08.159 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:08.159 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:08.167 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:08.251 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:08.250+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:08.317 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12442 3.9 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.107903088, 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036923198999999997, 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12442 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204567+0000", 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.204568+0000", 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6500000000000001e-06 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207214+0000", 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0026454019999999998 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207240+0000", 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6523e-05 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207254+0000", 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.43e-05 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207335+0000", 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0395000000000005e-05 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207518+0000", 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182819 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 7", 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208440+0000", 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00092235600000000004 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225462+0000", 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017022065999999999 2024-05-08T03:20:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241319+0000", 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015856677 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241446+0000", 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012764100000000001 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.241490+0000", 2024-05-08T03:20:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.337e-05 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23265 3.1a e55/28 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head v 55'298, mlcod=55'295)", 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.086832+0000", 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.2256380040000003, 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020384744, 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086832+0000", 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086832+0000", 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086833+0000", 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.513e-06 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086844+0000", 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1090000000000001e-05 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086864+0000", 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0055999999999999e-05 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086875+0000", 2024-05-08T03:20:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0651e-05 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086951+0000", 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6365000000000005e-05 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.086974+0000", 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2289e-05 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.107152+0000", 2024-05-08T03:20:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020178048000000001 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.107216+0000", 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4732000000000003e-05 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30814 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 204800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.122592+0000", 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.189877248, 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018622856, 2024-05-08T03:20:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30814 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122592+0000", 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122592+0000", 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122593+0000", 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.16e-07 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122594+0000", 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.415e-06 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122600+0000", 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7270000000000004e-06 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122608+0000", 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.072e-06 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141132+0000", 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018524031 2024-05-08T03:20:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141175+0000", 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2651000000000002e-05 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141215+0000", 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0744000000000002e-05 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30815 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 233472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.122615+0000", 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18985517199999999, 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018771659, 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30815 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122615+0000", 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122615+0000", 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122615+0000", 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9400000000000001e-07 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122622+0000", 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2280000000000001e-06 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122628+0000", 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2169999999999999e-06 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.122633+0000", 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8740000000000003e-06 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141243+0000", 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018609929000000001 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141312+0000", 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8899000000000006e-05 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141386+0000", 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4117999999999997e-05 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30839 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 49152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123009+0000", 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18946049700000001, 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018700676999999999, 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30839 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123009+0000", 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123009+0000", 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123010+0000", 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0900000000000002e-07 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123012+0000", 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1339999999999998e-06 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123018+0000", 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2029999999999997e-06 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123023+0000", 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7940000000000002e-06 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141426+0000", 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.01840321 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141508+0000", 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2328000000000002e-05 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141710+0000", 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020159900000000001 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30840 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 77824~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123029+0000", 2024-05-08T03:20:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18944040000000001, 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019051743999999999, 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30840 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123029+0000", 2024-05-08T03:20:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123029+0000", 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123030+0000", 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4000000000000002e-07 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123032+0000", 2024-05-08T03:20:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.103e-06 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123038+0000", 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0780000000000001e-06 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123043+0000", 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8969999999999999e-06 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141763+0000", 2024-05-08T03:20:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018719814000000001 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.141848+0000", 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5562000000000005e-05 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142081+0000", 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00023274999999999999 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30841 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 106496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123049+0000", 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18942097799999999, 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019180559999999999, 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30841 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123049+0000", 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123049+0000", 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123049+0000", 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:20:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123051+0000", 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123057+0000", 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000001e-06 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123062+0000", 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9230000000000001e-06 2024-05-08T03:20:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142129+0000", 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019067025000000001 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142182+0000", 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3878999999999997e-05 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142229+0000", 2024-05-08T03:20:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6807000000000001e-05 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30842 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 135168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123068+0000", 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18940196400000001, 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019300400999999998, 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30842 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123068+0000", 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123068+0000", 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123068+0000", 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5299999999999999e-07 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123077+0000", 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123084+0000", 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3289999999999999e-06 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123096+0000", 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2007999999999999e-05 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142264+0000", 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019168517 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142326+0000", 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2058999999999997e-05 2024-05-08T03:20:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142368+0000", 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1893e-05 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30843 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 163840~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123103+0000", 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18936636600000001, 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019391222999999999, 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30843 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123103+0000", 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123103+0000", 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123104+0000", 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9299999999999999e-07 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123106+0000", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5689999999999999e-06 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123113+0000", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5520000000000001e-06 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123118+0000", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9869999999999999e-06 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142399+0000", 2024-05-08T03:20:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019280971000000001 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142452+0000", 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3025999999999997e-05 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142495+0000", 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2725000000000002e-05 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30844 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123124+0000", 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18934568099999999, 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019521751, 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30844 2024-05-08T03:20:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123124+0000", 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123124+0000", 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123124+0000", 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6800000000000001e-07 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123126+0000", 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9860000000000001e-06 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123132+0000", 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8239999999999996e-06 2024-05-08T03:20:08.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123145+0000", 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2536e-05 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142534+0000", 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019389566 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142587+0000", 2024-05-08T03:20:08.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2797000000000003e-05 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142646+0000", 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8674000000000001e-05 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30845 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 221184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123154+0000", 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18931535099999999, 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019670399000000002, 2024-05-08T03:20:08.363 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30845 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123154+0000", 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123154+0000", 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123155+0000", 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9900000000000001e-07 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123159+0000", 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123165+0000", 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1299999999999998e-06 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123170+0000", 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6439999999999999e-06 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142687+0000", 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019517386000000001 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142768+0000", 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0344000000000002e-05 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142825+0000", 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7135000000000001e-05 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.367 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30846 3.1f 3:fa23decb:::rbd_data.1182224e9f13.0000000000000007:15 [read 249856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123176+0000", 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18929390400000001, 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019872349000000001, 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30846 2024-05-08T03:20:08.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123176+0000", 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123176+0000", 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123176+0000", 2024-05-08T03:20:08.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7899999999999999e-07 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123178+0000", 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0109999999999999e-06 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123184+0000", 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0499999999999997e-06 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123189+0000", 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6160000000000004e-06 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142874+0000", 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019684664000000001 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.142958+0000", 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4697999999999994e-05 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143048+0000", 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9931000000000003e-05 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30855 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 57344~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123377+0000", 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18909229299999999, 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019857234000000001, 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.372 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30855 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123377+0000", 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123377+0000", 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123378+0000", 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9000000000000002e-07 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123380+0000", 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.08e-06 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123387+0000", 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6499999999999999e-06 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123391+0000", 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6619999999999996e-06 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143094+0000", 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019703064999999999 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143138+0000", 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3643000000000002e-05 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143235+0000", 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6743999999999994e-05 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.376 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30856 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 86016~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123398+0000", 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18907148100000001, 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019985492000000001, 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.377 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30856 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123398+0000", 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123398+0000", 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123399+0000", 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8299999999999998e-07 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123401+0000", 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.018e-06 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123406+0000", 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3229999999999997e-06 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123410+0000", 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4170000000000002e-06 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143273+0000", 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019862999999999999 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143322+0000", 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8327e-05 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143384+0000", 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2024000000000001e-05 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30857 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 114688~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123417+0000", 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.189052469, 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020119689999999999, 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.381 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30857 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123417+0000", 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123417+0000", 2024-05-08T03:20:08.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123418+0000", 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0699999999999998e-07 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123420+0000", 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0940000000000002e-06 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123426+0000", 2024-05-08T03:20:08.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1809999999999998e-06 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123431+0000", 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2859999999999999e-06 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143425+0000", 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019993562999999999 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143495+0000", 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9796999999999996e-05 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143537+0000", 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2361999999999999e-05 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30858 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 143360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123438+0000", 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.189031849, 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020292233, 2024-05-08T03:20:08.385 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30858 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123438+0000", 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123438+0000", 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123438+0000", 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.77e-07 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123440+0000", 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0329999999999998e-06 2024-05-08T03:20:08.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123446+0000", 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9750000000000004e-06 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123451+0000", 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9119999999999997e-06 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143573+0000", 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020122145000000001 2024-05-08T03:20:08.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143656+0000", 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2384000000000003e-05 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143730+0000", 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4406999999999994e-05 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.389 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30859 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 172032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123457+0000", 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18901263900000001, 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020460288, 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30859 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123457+0000", 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123457+0000", 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123457+0000", 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123459+0000", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7770000000000001e-06 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123465+0000", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.665e-06 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123469+0000", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5329999999999996e-06 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143774+0000", 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020304352000000001 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143855+0000", 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1475000000000002e-05 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143917+0000", 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2117000000000002e-05 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30860 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 200704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123742+0000", 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.188727217, 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020303341999999999, 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30860 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.394 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123742+0000", 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123742+0000", 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123743+0000", 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:20:08.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123746+0000", 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.128e-06 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123756+0000", 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6500000000000008e-06 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123762+0000", 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2190000000000001e-06 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143951+0000", 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020188608 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.143992+0000", 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1403000000000003e-05 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144046+0000", 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3786000000000003e-05 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.397 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30861 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 229376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123779+0000", 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18869091299999999, 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020385024000000002, 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30861 2024-05-08T03:20:08.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123779+0000", 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123779+0000", 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123779+0000", 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5400000000000001e-07 2024-05-08T03:20:08.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123788+0000", 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5019999999999992e-06 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123796+0000", 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1370000000000002e-06 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123802+0000", 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:08.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144073+0000", 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020271166 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144121+0000", 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7580000000000002e-05 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144164+0000", 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3297999999999998e-05 2024-05-08T03:20:08.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30862 3.1f 3:fa5e63f4:::rbd_data.1182224e9f13.0000000000000009:15 [read 258048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.123810+0000", 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.18865942199999999, 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020487854, 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30862 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123810+0000", 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123810+0000", 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123811+0000", 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8400000000000005e-07 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123813+0000", 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3690000000000001e-06 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123820+0000", 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9759999999999998e-06 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.123834+0000", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.384e-05 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144191+0000", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020356893000000001 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144244+0000", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2799000000000001e-05 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.144298+0000", 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4493000000000002e-05 2024-05-08T03:20:08.405 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.406 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.406 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.406 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:08.407 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:08.408 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:08.406+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:08.508 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:08.506+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:08.553 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:08.553 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:08.553 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:08.553 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:08.562 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:08.645 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:08.644+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:08.706 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:08.706 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:08.706 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:08.706 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:08.707 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:08.709 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:08.708+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:08.715 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:08.810 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:08.808+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:08.836 INFO:tasks.thrashosds.thrasher:No PGs found for osd.5 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.8635146079999991, 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.8634471440000002, 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.8036128120000008, 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.7371047990000008, 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.7105671529999995, 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.6826553850000003, 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23281 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'92, mlcod=53'88)", 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.6331098659999999, 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074279843999999998, 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226785+0000", 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.299e-06 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226794+0000", 2024-05-08T03:20:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5040000000000008e-06 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226810+0000", 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5781e-05 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226820+0000", 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9310000000000003e-06 2024-05-08T03:20:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262249+0000", 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.035428501000000001 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262269+0000", 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0077000000000001e-05 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301028+0000", 2024-05-08T03:20:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.038759184000000002 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301064+0000", 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5567e-05 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26865 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4288512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.965497, 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041924771999999999, 2024-05-08T03:20:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26865 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894399+0000", 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6309999999999999e-06 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894403+0000", 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8559999999999998e-06 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894408+0000", 2024-05-08T03:20:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9259999999999999e-06 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936202+0000", 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041793779000000003 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936255+0000", 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3072000000000003e-05 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936321+0000", 2024-05-08T03:20:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6128000000000001e-05 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26866 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4460544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9654805629999998, 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042116690999999998, 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26866 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0400000000000002e-07 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894415+0000", 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5990000000000001e-06 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894420+0000", 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9119999999999997e-06 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894424+0000", 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7299999999999999e-06 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936364+0000", 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041939999999999998 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936447+0000", 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3293000000000004e-05 2024-05-08T03:20:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936530+0000", 2024-05-08T03:20:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2853000000000001e-05 2024-05-08T03:20:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26867 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4509696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9654657660000003, 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042262936000000001, 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26867 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896093+0000", 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016655789999999999 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896098+0000", 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8180000000000003e-06 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896109+0000", 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1109e-05 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896118+0000", 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8489999999999995e-06 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936561+0000", 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040442845999999998 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936610+0000", 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9286000000000002e-05 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936691+0000", 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0449000000000002e-05 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26874 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4386816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.963638918, 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041925208999999998, 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26874 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896258+0000", 2024-05-08T03:20:08.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9979999999999999e-06 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896261+0000", 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1470000000000001e-06 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896265+0000", 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9419999999999997e-06 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938052+0000", 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041787283000000001 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938114+0000", 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2020000000000006e-05 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938180+0000", 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5611999999999995e-05 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.919 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26875 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4411392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9636235559999999, 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.04205014, 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26875 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.920 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:08.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9200000000000002e-07 2024-05-08T03:20:08.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896272+0000", 2024-05-08T03:20:08.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6789999999999999e-06 2024-05-08T03:20:08.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896277+0000", 2024-05-08T03:20:08.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8289999999999998e-06 2024-05-08T03:20:08.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896281+0000", 2024-05-08T03:20:08.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.839e-06 2024-05-08T03:20:08.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938214+0000", 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041933639000000002 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938260+0000", 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5510000000000003e-05 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938320+0000", 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0251999999999998e-05 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26876 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4435968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.963607036, 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042168543000000003, 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.928 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26876 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:08.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896289+0000", 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9470000000000002e-06 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896292+0000", 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5470000000000001e-06 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896296+0000", 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.574e-06 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938354+0000", 2024-05-08T03:20:08.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042057861000000002 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938398+0000", 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4221999999999998e-05 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938455+0000", 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7037999999999998e-05 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26877 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4608000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9635941050000003, 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042279939000000002, 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26877 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896300+0000", 2024-05-08T03:20:08.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896301+0000", 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.159e-06 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896304+0000", 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.881e-06 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896307+0000", 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938478+0000", 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042171023000000002 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938524+0000", 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6109999999999997e-05 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938579+0000", 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5448000000000002e-05 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26878 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4632576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9635835960000003, 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042399394, 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.937 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26878 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898092+0000", 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0017816349999999999 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898096+0000", 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7720000000000002e-06 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898110+0000", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3764000000000001e-05 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898118+0000", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1659999999999994e-06 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938611+0000", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040492570999999998 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938654+0000", 2024-05-08T03:20:08.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3133000000000002e-05 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938709+0000", 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5352999999999997e-05 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26892 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4853760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9614372339999999, 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041959164, 2024-05-08T03:20:08.941 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26892 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.942 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898457+0000", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3399999999999999e-07 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898459+0000", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2960000000000001e-06 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898466+0000", 2024-05-08T03:20:08.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2400000000000001e-06 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898471+0000", 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.532e-06 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940319+0000", 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041847652999999999 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940360+0000", 2024-05-08T03:20:08.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0952999999999999e-05 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940415+0000", 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5955999999999998e-05 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26893 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4878336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9614167849999999, 2024-05-08T03:20:08.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042048864999999998, 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26893 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898479+0000", 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:08.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898489+0000", 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6900000000000004e-06 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898493+0000", 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5709999999999999e-06 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940438+0000", 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041944253000000001 2024-05-08T03:20:08.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940475+0000", 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7512000000000003e-05 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940526+0000", 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0599000000000001e-05 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.949 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26894 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9613917199999999, 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042154042000000003, 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26894 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900091+0000", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015893400000000001 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900096+0000", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900110+0000", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3773e-05 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900119+0000", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3880000000000006e-06 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940559+0000", 2024-05-08T03:20:08.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040439875 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940601+0000", 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1950999999999999e-05 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940656+0000", 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4960999999999998e-05 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.953 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26909 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9594283289999996, 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041961032000000002, 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26909 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900466+0000", 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3799999999999998e-07 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900468+0000", 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.447e-06 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900475+0000", 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9380000000000003e-06 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900479+0000", 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2030000000000002e-06 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942337+0000", 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041857710999999999 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942376+0000", 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9533999999999998e-05 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942426+0000", 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9861000000000003e-05 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.957 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26910 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 5566464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9594083300000005, 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042062763000000003, 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26910 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.958 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902091+0000", 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001605778 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902096+0000", 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5759999999999999e-06 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902108+0000", 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2018000000000001e-05 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902116+0000", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8429999999999999e-06 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942456+0000", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040339331999999999 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942496+0000", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9774999999999998e-05 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:08.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942548+0000", 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2441000000000002e-05 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:08.962 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:08.964 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:08.964 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:08.962+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:09.065 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:09.063+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:09.076 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:09.076 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:09.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:09.166 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:09.164+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:09.201 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:09.202 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:09.202 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:09.202 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:09.202 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:09.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:09.267 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:09.266+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.455778252999998, 2024-05-08T03:20:09.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:09.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:20:09.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:20:09.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.455436075000001, 2024-05-08T03:20:09.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:20:09.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:20:09.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.455397243, 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.338 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:20:09.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:20:09.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.455351210000003, 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.344 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.345 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:20:09.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:20:09.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.349 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.454334017999997, 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.350 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:09.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:20:09.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:20:09.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.355 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.454297767, 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:20:09.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:20:09.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:20:09.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:20:09.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.454270467999997, 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:20:09.362 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:20:09.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.454240775000002, 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.368 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:20:09.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:20:09.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.373 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.454215259999998, 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:20:09.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:09.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:20:09.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:20:09.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.444101093999997, 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.380 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:20:09.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:20:09.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:20:09.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191762249, 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:20:09.385 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191743918, 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:20:09.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:20:09.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191719456999998, 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.394 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:20:09.396 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:20:09.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:09.398 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191693114000003, 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:09.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.402 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191675723000003, 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.403 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191658441999998, 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.407 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:20:09.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:20:09.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191631004999998, 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:20:09.411 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:20:09.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:09.415 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191613046000001, 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:20:09.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:20:09.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191595960000001, 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:20:09.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.191578976000002, 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:09.424 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:09.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:09.428 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-05-08T03:20:09.430 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:09.431 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:09.429+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:09.530 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:09.529+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:09.575 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:09.575 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:09.584 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:09.624 INFO:teuthology.orchestra.run.smithi177.stdout:2125 punch from 0x2b226d to 0x2b6893, (0x4626 bytes) 2024-05-08T03:20:09.627 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:09.627 INFO:teuthology.orchestra.run.smithi177.stdout:2127 punch from 0x40cd430 to 0x40ce2cd, (0xe9d bytes) 2024-05-08T03:20:09.632 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:09.630+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:09.632 INFO:teuthology.orchestra.run.smithi177.stdout:2129 trunc from 0x4c40342 to 0xe6ec6c 2024-05-08T03:20:09.667 INFO:teuthology.orchestra.run.smithi177.stdout:2130 write 0x2a11091 thru 0x2a1a581 (0x94f1 bytes) 2024-05-08T03:20:09.672 INFO:teuthology.orchestra.run.smithi177.stdout:2132 read 0x6b35ed thru 0x6b61d3 (0x2be7 bytes) 2024-05-08T03:20:09.688 INFO:teuthology.orchestra.run.smithi177.stdout:2133 write 0xafe3669 thru 0xafe5a6a (0x2402 bytes) 2024-05-08T03:20:09.691 INFO:teuthology.orchestra.run.smithi177.stdout:2134 punch from 0x9794fe to 0x980920, (0x7422 bytes) 2024-05-08T03:20:09.691 INFO:teuthology.orchestra.run.smithi177.stdout:2135 read 0x3b52176 thru 0x3b560fc (0x3f87 bytes) 2024-05-08T03:20:09.691 INFO:teuthology.orchestra.run.smithi177.stdout:2136 punch from 0x703f839 to 0x7048045, (0x880c bytes) 2024-05-08T03:20:09.691 INFO:teuthology.orchestra.run.smithi177.stdout:2137 write 0x961bb14 thru 0x96224ba (0x69a7 bytes) 2024-05-08T03:20:09.691 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:09.691 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:09.694 INFO:teuthology.orchestra.run.smithi177.stdout:2138 write 0x90b4375 thru 0x90be298 (0x9f24 bytes) 2024-05-08T03:20:09.698 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:09.698 INFO:teuthology.orchestra.run.smithi177.stdout:2140 write 0x7786c2f thru 0x7793fda (0xd3ac bytes) 2024-05-08T03:20:09.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:09.707 INFO:teuthology.orchestra.run.smithi177.stdout:2141 write 0xb7786a7 thru 0xb785e4b (0xd7a5 bytes) 2024-05-08T03:20:09.724 INFO:teuthology.orchestra.run.smithi177.stdout:2142 trunc from 0xb785e4c to 0xbe011a 2024-05-08T03:20:09.732 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:09.731+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:09.761 INFO:teuthology.orchestra.run.smithi177.stdout:2143 writesame 0xd8de9ca thru 0xd8eda74 data_size 0xb(0xf0ab bytes) 2024-05-08T03:20:09.762 INFO:teuthology.orchestra.run.smithi177.stdout:2144 read 0xaa133b9 thru 0xaa15a77 (0x26bf bytes) 2024-05-08T03:20:09.762 INFO:teuthology.orchestra.run.smithi177.stdout:2145 write 0x21cbf5d thru 0x21d6107 (0xa1ab bytes) 2024-05-08T03:20:09.765 INFO:teuthology.orchestra.run.smithi177.stdout:2146 read 0xa17262e thru 0xa175c1f (0x35f2 bytes) 2024-05-08T03:20:09.765 INFO:teuthology.orchestra.run.smithi177.stdout:2147 write 0x7431c57 thru 0x743b3ed (0x9797 bytes) 2024-05-08T03:20:09.769 INFO:teuthology.orchestra.run.smithi177.stdout:2148 write 0x1e0f0aa thru 0x1e0ff45 (0xe9c bytes) 2024-05-08T03:20:09.771 INFO:teuthology.orchestra.run.smithi177.stdout:2149 read 0xe752c thru 0xf596c (0xe441 bytes) 2024-05-08T03:20:09.773 INFO:teuthology.orchestra.run.smithi177.stdout:2150 punch from 0xce011f1 to 0xce10fde, (0xfded bytes) 2024-05-08T03:20:09.781 INFO:teuthology.orchestra.run.smithi177.stdout:2152 trunc from 0xd8eda75 to 0x6cc37bc 2024-05-08T03:20:09.789 INFO:teuthology.orchestra.run.smithi177.stdout:2155 read 0x44b2a81 thru 0x44b598a (0x2f0a bytes) 2024-05-08T03:20:09.795 INFO:teuthology.orchestra.run.smithi177.stdout:2157 trunc from 0x6cc37bc to 0xbc2ed95 2024-05-08T03:20:09.806 INFO:teuthology.orchestra.run.smithi177.stdout:2158 write 0xd775c4a thru 0xd77d8df (0x7c96 bytes) 2024-05-08T03:20:09.809 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:09.809 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:09.811 INFO:teuthology.orchestra.run.smithi177.stdout:2160 write 0xc8a67e2 thru 0xc8a8945 (0x2164 bytes) 2024-05-08T03:20:09.814 INFO:teuthology.orchestra.run.smithi177.stdout:2162 read 0xa4d3d62 thru 0xa4dce68 (0x9107 bytes) 2024-05-08T03:20:09.814 INFO:teuthology.orchestra.run.smithi177.stdout:2163 punch from 0xb982b84 to 0xb98e6bc, (0xbb38 bytes) 2024-05-08T03:20:09.814 INFO:teuthology.orchestra.run.smithi177.stdout:2164 read 0x34c9c11 thru 0x34ca79a (0xb8a bytes) 2024-05-08T03:20:09.814 INFO:teuthology.orchestra.run.smithi177.stdout:2165 writesame 0x14a76fd thru 0x14b63be data_size 0x1d(0xecc2 bytes) 2024-05-08T03:20:09.818 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:09.819 INFO:teuthology.orchestra.run.smithi177.stdout:2166 write 0xc8bf7e5 thru 0xc8c8f57 (0x9773 bytes) 2024-05-08T03:20:09.824 INFO:teuthology.orchestra.run.smithi177.stdout:2167 writesame 0xd39ee61 thru 0xd3ae0ba data_size 0x43(0xf25a bytes) 2024-05-08T03:20:09.830 INFO:teuthology.orchestra.run.smithi177.stdout:2168 trunc from 0xd77d8e0 to 0xb552bcd 2024-05-08T03:20:09.840 INFO:teuthology.orchestra.run.smithi177.stdout:2170 clone 21 order 21 su 1048576 sc 8 2024-05-08T03:20:09.862 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:09.861+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:09.923 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:09.924 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:09.932 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:09.933 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:09.932+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:10.035 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:10.034+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:10.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:20:10.080 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:10.080 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:10.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:10.115 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone20 from 0xb552bcd (overlap 0xbe011a) to 0xab00fd 2024-05-08T03:20:10.136 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:10.135+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:10.179 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-05-08T03:20:10.198 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:10.198 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:10.207 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:10.250 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:10.249+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:10.287 INFO:teuthology.orchestra.run.smithi177.stdout:2173 read 0x448f1a0 thru 0x4499556 (0xa3b7 bytes) 2024-05-08T03:20:10.288 INFO:teuthology.orchestra.run.smithi177.stdout:2174 punch from 0x72b5e47 to 0x72c47d2, (0xe98b bytes) 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:2176 punch from 0x3d556ae to 0x3d5c237, (0x6b89 bytes) 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:2177 punch from 0xa248801 to 0xa256914, (0xe113 bytes) 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:2180 read 0x72d3a75 thru 0x72d8060 (0x45ec bytes) 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:2183 read 0xaa88103 thru 0xaa97e99 (0xfd97 bytes) 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:2184 punch from 0x771e723 to 0x7726997, (0x8274 bytes) 2024-05-08T03:20:10.293 INFO:teuthology.orchestra.run.smithi177.stdout:2185 writesame 0x336f93e thru 0x33735e0 data_size 0x13(0x3ca3 bytes) 2024-05-08T03:20:10.296 INFO:teuthology.orchestra.run.smithi177.stdout:2186 punch from 0x4107fb1 to 0x41149aa, (0xc9f9 bytes) 2024-05-08T03:20:10.302 INFO:teuthology.orchestra.run.smithi177.stdout:2187 write 0xbd3fba2 thru 0xbd4f679 (0xfad8 bytes) 2024-05-08T03:20:10.313 INFO:teuthology.orchestra.run.smithi177.stdout:2188 write 0xd030447 thru 0xd0370e0 (0x6c9a bytes) 2024-05-08T03:20:10.316 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.319 INFO:teuthology.orchestra.run.smithi177.stdout:2190 trunc from 0xd0370e1 to 0x9e12871 2024-05-08T03:20:10.331 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:10.331 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:10.331 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:10.331 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:10.338 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:10.336+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:10.343 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:10.344 INFO:teuthology.orchestra.run.smithi177.stdout:2191 write 0xb60a921 thru 0xb60e27b (0x395b bytes) 2024-05-08T03:20:10.347 INFO:teuthology.orchestra.run.smithi177.stdout:2192 read 0x5a187de thru 0x5a1b4b3 (0x2cd6 bytes) 2024-05-08T03:20:10.347 INFO:teuthology.orchestra.run.smithi177.stdout:2193 write 0x57eb6e9 thru 0x57f77eb (0xc103 bytes) 2024-05-08T03:20:10.350 INFO:teuthology.orchestra.run.smithi177.stdout:2194 write 0x91b5d1e thru 0x91c2a33 (0xcd16 bytes) 2024-05-08T03:20:10.354 INFO:teuthology.orchestra.run.smithi177.stdout:2195 trunc from 0xb60e27c to 0xbd91e78 2024-05-08T03:20:10.359 INFO:teuthology.orchestra.run.smithi177.stdout:2196 punch from 0x44bd206 to 0x44c9210, (0xc00a bytes) 2024-05-08T03:20:10.365 INFO:teuthology.orchestra.run.smithi177.stdout:2197 trunc from 0xbd91e78 to 0x741f59e 2024-05-08T03:20:10.385 INFO:teuthology.orchestra.run.smithi177.stdout:2198 read 0xc260c1 thru 0xc2ec4f (0x8b8f bytes) 2024-05-08T03:20:10.385 INFO:teuthology.orchestra.run.smithi177.stdout:2200 read 0x36d1c08 thru 0x36d783b (0x5c34 bytes) 2024-05-08T03:20:10.396 INFO:teuthology.orchestra.run.smithi177.stdout:2202 write 0xc6a4206 thru 0xc6b1ad3 (0xd8ce bytes) 2024-05-08T03:20:10.399 INFO:teuthology.orchestra.run.smithi177.stdout:2203 punch from 0x22c2db5 to 0x22cc092, (0x92dd bytes) 2024-05-08T03:20:10.410 INFO:teuthology.orchestra.run.smithi177.stdout:2204 trunc from 0xc6b1ad4 to 0x4006d46 2024-05-08T03:20:10.437 INFO:teuthology.orchestra.run.smithi177.stdout:2205 trunc from 0x4006d46 to 0xe3bb697 2024-05-08T03:20:10.439 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:10.437+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:10.442 INFO:teuthology.orchestra.run.smithi177.stdout:2206 write 0x3616bfd thru 0x361e10c (0x7510 bytes) 2024-05-08T03:20:10.445 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.445 INFO:teuthology.orchestra.run.smithi177.stdout:2208 punch from 0x2bc590 to 0x2bf1bb, (0x2c2b bytes) 2024-05-08T03:20:10.445 INFO:teuthology.orchestra.run.smithi177.stdout:2210 writesame 0x799e4ea thru 0x79ad1a7 data_size 0x2a(0xecbe bytes) 2024-05-08T03:20:10.448 INFO:teuthology.orchestra.run.smithi177.stdout:2211 write 0x662e061 thru 0x6635175 (0x7115 bytes) 2024-05-08T03:20:10.452 INFO:teuthology.orchestra.run.smithi177.stdout:2214 read 0x2f545c9 thru 0x2f636be (0xf0f6 bytes) 2024-05-08T03:20:10.452 INFO:teuthology.orchestra.run.smithi177.stdout:2215 punch from 0x2ccf34e to 0x2cd2311, (0x2fc3 bytes) 2024-05-08T03:20:10.452 INFO:teuthology.orchestra.run.smithi177.stdout:2216 punch from 0x493760e to 0x493c8be, (0x52b0 bytes) 2024-05-08T03:20:10.452 INFO:teuthology.orchestra.run.smithi177.stdout:2217 write 0x8083a90 thru 0x808963b (0x5bac bytes) 2024-05-08T03:20:10.455 INFO:teuthology.orchestra.run.smithi177.stdout:2218 write 0x5656f20 thru 0x565c4e1 (0x55c2 bytes) 2024-05-08T03:20:10.458 INFO:teuthology.orchestra.run.smithi177.stdout:2219 punch from 0x947a51e to 0x947fb2b, (0x560d bytes) 2024-05-08T03:20:10.459 INFO:teuthology.orchestra.run.smithi177.stdout:2221 punch from 0xce3bbbb to 0xce468af, (0xacf4 bytes) 2024-05-08T03:20:10.459 INFO:teuthology.orchestra.run.smithi177.stdout:2223 punch from 0xd462a03 to 0xd46495e, (0x1f5b bytes) 2024-05-08T03:20:10.459 INFO:teuthology.orchestra.run.smithi177.stdout:2225 writesame 0x6a071ad thru 0x6a14dd3 data_size 0x1(0xdc27 bytes) 2024-05-08T03:20:10.473 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:10.474 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:10.474 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:10.474 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:10.474 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:10.483 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:10.489 INFO:teuthology.orchestra.run.smithi177.stdout:2226 read 0xb23b1bc thru 0xb23e8f2 (0x3737 bytes) 2024-05-08T03:20:10.489 INFO:teuthology.orchestra.run.smithi177.stdout:2227 write 0xc9dc51 thru 0xcabf96 (0xe346 bytes) 2024-05-08T03:20:10.492 INFO:teuthology.orchestra.run.smithi177.stdout:2228 read 0x3ee3dec thru 0x3eeaf84 (0x7199 bytes) 2024-05-08T03:20:10.492 INFO:teuthology.orchestra.run.smithi177.stdout:2230 read 0x47548a7 thru 0x475f905 (0xb05f bytes) 2024-05-08T03:20:10.493 INFO:teuthology.orchestra.run.smithi177.stdout:2232 write 0x4a11784 thru 0x4a20b3a (0xf3b7 bytes) 2024-05-08T03:20:10.496 INFO:teuthology.orchestra.run.smithi177.stdout:2233 trunc from 0xe3bb697 to 0xe4bc838 2024-05-08T03:20:10.502 INFO:teuthology.orchestra.run.smithi177.stdout:2234 trunc from 0xe4bc838 to 0xe1de993 2024-05-08T03:20:10.506 INFO:teuthology.orchestra.run.smithi177.stdout:2236 punch from 0x7caffc6 to 0x7cbb901, (0xb93b bytes) 2024-05-08T03:20:10.506 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.507 INFO:teuthology.orchestra.run.smithi177.stdout:2239 read 0xc424056 thru 0xc424bf1 (0xb9c bytes) 2024-05-08T03:20:10.507 INFO:teuthology.orchestra.run.smithi177.stdout:2240 writesame 0xbf37007 thru 0xbf3ff60 data_size 0x2(0x8f5a bytes) 2024-05-08T03:20:10.518 INFO:teuthology.orchestra.run.smithi177.stdout:2241 write 0x48b2d24 thru 0x48c0583 (0xd860 bytes) 2024-05-08T03:20:10.519 INFO:teuthology.orchestra.run.smithi177.stdout:2242 writesame 0x1710f8c thru 0x171a443 data_size 0x4(0x94b8 bytes) 2024-05-08T03:20:10.526 INFO:teuthology.orchestra.run.smithi177.stdout:2243 write 0x78c1a3f thru 0x78ccf67 (0xb529 bytes) 2024-05-08T03:20:10.529 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.531 INFO:teuthology.orchestra.run.smithi177.stdout:2248 trunc from 0xe1de993 to 0xc2c18b7 2024-05-08T03:20:10.536 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.540 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:10.538+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:10.544 INFO:teuthology.orchestra.run.smithi177.stdout:2250 write 0xd4be6cc thru 0xd4ce4f5 (0xfe2a bytes) 2024-05-08T03:20:10.546 INFO:teuthology.orchestra.run.smithi177.stdout:2251 punch from 0x3a2e5ec to 0x3a37b6e, (0x9582 bytes) 2024-05-08T03:20:10.546 INFO:teuthology.orchestra.run.smithi177.stdout:2252 writesame 0x746a6da thru 0x7476db1 data_size 0x24(0xc6d8 bytes) 2024-05-08T03:20:10.557 INFO:teuthology.orchestra.run.smithi177.stdout:2253 trunc from 0xd4ce4f6 to 0x73ce733 2024-05-08T03:20:10.581 INFO:teuthology.orchestra.run.smithi177.stdout:2255 write 0xa6ef55f thru 0xa6fcfdd (0xda7f bytes) 2024-05-08T03:20:10.584 INFO:teuthology.orchestra.run.smithi177.stdout:2256 write 0x2d2c409 thru 0x2d399ba (0xd5b2 bytes) 2024-05-08T03:20:10.592 INFO:teuthology.orchestra.run.smithi177.stdout:2258 trunc from 0xa6fcfde to 0xe2a5929 2024-05-08T03:20:10.597 INFO:teuthology.orchestra.run.smithi177.stdout:2259 write 0xa11661d thru 0xa11a2e0 (0x3cc4 bytes) 2024-05-08T03:20:10.600 INFO:teuthology.orchestra.run.smithi177.stdout:2264 writesame 0xa36a020 thru 0xa36b90a data_size 0x1(0x18eb bytes) 2024-05-08T03:20:10.604 INFO:teuthology.orchestra.run.smithi177.stdout:2266 write 0xadcb32d thru 0xadcf055 (0x3d29 bytes) 2024-05-08T03:20:10.607 INFO:teuthology.orchestra.run.smithi177.stdout:2267 read 0xae442e6 thru 0xae471b8 (0x2ed3 bytes) 2024-05-08T03:20:10.608 INFO:teuthology.orchestra.run.smithi177.stdout:2268 write 0x6f4fe12 thru 0x6f5d337 (0xd526 bytes) 2024-05-08T03:20:10.610 INFO:teuthology.orchestra.run.smithi177.stdout:2269 write 0x176e63f thru 0x176f6d8 (0x109a bytes) 2024-05-08T03:20:10.623 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:10.623 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:10.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:10.623 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:10.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.623 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.420770965999999, 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:10.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.420761515999999, 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:10.631 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.420748779999997, 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:10.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.420738204000003, 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.636 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:10.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:10.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.420728816999997, 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:10.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:10.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:10.644 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.420719636999998, 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.645 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:10.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189894602000003, 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.649 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:10.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:10.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189862365000003, 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:10.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189849035000002, 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.658 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.662 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189834060999999, 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:10.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:10.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:10.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189822751999998, 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.667 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:10.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189805083000003, 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:10.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:10.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189789885000003, 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.676 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:10.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:10.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.189775791000002, 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.680 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:10.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:10.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:10.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.414821818, 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:10.684 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:10.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:10.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.491928016000003, 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.690 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:10.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:10.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.491914528000002, 2024-05-08T03:20:10.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:10.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.698 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.491885547000003, 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.699 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:10.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.702 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.491858940999997, 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:10.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:10.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.706 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.059110680000003, 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:10.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:10.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:10.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:10.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:10.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.712 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:10.713 INFO:teuthology.orchestra.run.smithi177.stdout:2270 trunc from 0xe2a5929 to 0x44240be 2024-05-08T03:20:10.714 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2271 write 0xd7ddc94 thru 0xd7e2509 (0x4876 bytes) 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2272 read 0x192fe0d thru 0x193ac7f (0xae73 bytes) 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2274 trunc from 0xd7e250a to 0x46872f2 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2276 write 0x1a1fc43 thru 0x1a2f83c (0xfbfa bytes) 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2277 punch from 0x463f400 to 0x463fcdf, (0x8df bytes) 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2278 read 0x22234cf thru 0x222d581 (0xa0b3 bytes) 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2279 writesame 0xee3d17a thru 0xee4c87d data_size 0x2(0xf704 bytes) 2024-05-08T03:20:10.716 INFO:teuthology.orchestra.run.smithi177.stdout:2281 read 0xd32be9d thru 0xd32edbe (0x2f22 bytes) 2024-05-08T03:20:10.717 INFO:teuthology.orchestra.run.smithi177.stdout:2282 write 0xfe1cc1 thru 0xfee32c (0xc66c bytes) 2024-05-08T03:20:10.717 INFO:teuthology.orchestra.run.smithi177.stdout:2284 write 0x726fe5e thru 0x727eec4 (0xf067 bytes) 2024-05-08T03:20:10.717 INFO:teuthology.orchestra.run.smithi177.stdout:2285 read 0x3c3477b thru 0x3c382ea (0x3b70 bytes) 2024-05-08T03:20:10.717 INFO:teuthology.orchestra.run.smithi177.stdout:2286 writesame 0xcf04f6f thru 0xcf0c090 data_size 0x9(0x7122 bytes) 2024-05-08T03:20:10.717 INFO:teuthology.orchestra.run.smithi177.stdout:2287 write 0x3d704cd thru 0x3d7f016 (0xeb4a bytes) 2024-05-08T03:20:10.717 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:10.712+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:10.718 INFO:teuthology.orchestra.run.smithi177.stdout:2288 punch from 0x2d7769 to 0x2e470f, (0xcfa6 bytes) 2024-05-08T03:20:10.718 INFO:teuthology.orchestra.run.smithi177.stdout:2291 writesame 0xceeed27 thru 0xcef39c1 data_size 0x3(0x4c9b bytes) 2024-05-08T03:20:10.720 INFO:teuthology.orchestra.run.smithi177.stdout:2292 read 0x66c1acf thru 0x66cf7c9 (0xdcfb bytes) 2024-05-08T03:20:10.720 INFO:teuthology.orchestra.run.smithi177.stdout:2293 write 0xabe5c9d thru 0xabe9161 (0x34c5 bytes) 2024-05-08T03:20:10.735 INFO:teuthology.orchestra.run.smithi177.stdout:2294 trunc from 0xee4c87e to 0x4016f07 2024-05-08T03:20:10.763 INFO:teuthology.orchestra.run.smithi177.stdout:2298 write 0x812f0ab thru 0x813097b (0x18d1 bytes) 2024-05-08T03:20:10.766 INFO:teuthology.orchestra.run.smithi177.stdout:2300 read 0x688be56 thru 0x6895238 (0x93e3 bytes) 2024-05-08T03:20:10.771 INFO:teuthology.orchestra.run.smithi177.stdout:2302 trunc from 0x813097c to 0x44a4371 2024-05-08T03:20:10.782 INFO:teuthology.orchestra.run.smithi177.stdout:2304 read 0x3000b9e thru 0x3003f92 (0x33f5 bytes) 2024-05-08T03:20:10.782 INFO:teuthology.orchestra.run.smithi177.stdout:2306 read 0x92ed0a thru 0x93e734 (0xfa2b bytes) 2024-05-08T03:20:10.794 INFO:teuthology.orchestra.run.smithi177.stdout:2307 writesame 0x9c409d9 thru 0x9c41c08 data_size 0x18(0x1230 bytes) 2024-05-08T03:20:10.794 INFO:teuthology.orchestra.run.smithi177.stdout:2308 read 0x6bef531 thru 0x6bf0af0 (0x15c0 bytes) 2024-05-08T03:20:10.802 INFO:teuthology.orchestra.run.smithi177.stdout:2311 trunc from 0x9c41c09 to 0x30eed67 2024-05-08T03:20:10.814 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:10.812+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:10.823 INFO:teuthology.orchestra.run.smithi177.stdout:2312 trunc from 0x30eed67 to 0x351495c 2024-05-08T03:20:10.828 INFO:teuthology.orchestra.run.smithi177.stdout:2314 write 0x2663c24 thru 0x266dfe3 (0xa3c0 bytes) 2024-05-08T03:20:10.845 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:36428 3.4 3:20f5ca1e:::rbd_header.118285beaa44:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:10.844734+0000", 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00031886600000000001, 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033762000000000001, 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for sub ops", 2024-05-08T03:20:10.846 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 36428 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844734+0000", 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844734+0000", 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:10.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844735+0000", 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8999999999999995e-07 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844740+0000", 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6700000000000002e-06 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844753+0000", 2024-05-08T03:20:10.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2969000000000001e-05 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844761+0000", 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844809+0000", 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7533e-05 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.844889+0000", 2024-05-08T03:20:10.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9969999999999995e-05 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 2", 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:10.845071+0000", 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018194700000000001 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:20:10.850 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:10.851 INFO:teuthology.orchestra.run.smithi177.stdout:2315 write 0xb1c68bb thru 0xb1c935a (0x2aa0 bytes) 2024-05-08T03:20:10.852 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:10.852 INFO:teuthology.orchestra.run.smithi177.stdout:2319 punch from 0x9737477 to 0x973af40, (0x3ac9 bytes) 2024-05-08T03:20:10.856 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:10.865 INFO:teuthology.orchestra.run.smithi177.stdout:2321 trunc from 0xb1c935b to 0xb5c937 2024-05-08T03:20:10.888 INFO:teuthology.orchestra.run.smithi177.stdout:2322 read 0xf8c35 thru 0xfec75 (0x6041 bytes) 2024-05-08T03:20:10.889 INFO:teuthology.orchestra.run.smithi177.stdout:2323 clone 22 order 25 su 262144 sc 6 2024-05-08T03:20:10.915 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:10.913+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:10.972 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:10.972 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:10.973 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:10.973 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:10.973 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:10.981 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:11.022 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:11.021+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:11.064 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:20:11.117 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:11.115+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:11.127 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone21 from 0xb5c937 (overlap 0xb5c937) to 0x0 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 25.366738036000001, 2024-05-08T03:20:11.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:11.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:11.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.219503886999998, 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:11.168 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 11.029363719999999, 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:11.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:11.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:11.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:11.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:11.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 11.004623174000001, 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:11.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:11.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25865 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [sparse-read 0~19262493] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.682755974, 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053445962, 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.181 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25865 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469279+0000", 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.852e-06 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469291+0000", 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2098e-05 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469313+0000", 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2657999999999999e-05 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469326+0000", 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2782000000000001e-05 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469401+0000", 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4759999999999996e-05 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469495+0000", 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4000999999999997e-05 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.522722+0000", 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053226810999999999 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25866 3.1a 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.185 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6821347339999999, 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049237975000000003, 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25866 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469898+0000", 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.23e-07 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469902+0000", 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4839999999999998e-06 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469913+0000", 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0652999999999999e-05 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469926+0000", 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3010000000000001e-05 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470021+0000", 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5622e-05 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470105+0000", 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3938999999999997e-05 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.519135+0000", 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049029544000000001 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.189 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25871 3.16 3:692ca2ad:::rbd_data.118227bb2f9.0000000000000006:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.681947965, 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.047103610999999997, 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25871 2024-05-08T03:20:11.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:11.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.06e-07 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470092+0000", 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8310000000000001e-06 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470099+0000", 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0999999999999998e-06 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470108+0000", 2024-05-08T03:20:11.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3190000000000001e-06 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470162+0000", 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3618000000000001e-05 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470234+0000", 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2342999999999995e-05 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.517187+0000", 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046952894000000002 2024-05-08T03:20:11.193 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27760 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 557056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9728109659999999, 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046225914999999999, 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27760 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8900000000000001e-07 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179222+0000", 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.114e-06 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179225+0000", 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9189999999999999e-06 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179228+0000", 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.819e-06 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225336+0000", 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046108300999999997 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225388+0000", 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2057e-05 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225447+0000", 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8415999999999998e-05 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27761 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9728010669999998, 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046341465999999998, 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27761 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179232+0000", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179235+0000", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8770000000000002e-06 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179237+0000", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3690000000000001e-06 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225475+0000", 2024-05-08T03:20:11.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046237568 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225515+0000", 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0080000000000003e-05 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225572+0000", 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7351e-05 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27762 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 581632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9727919959999998, 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046463320000000002, 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27762 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.29e-07 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179241+0000", 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.263e-06 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179244+0000", 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7769999999999998e-06 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179246+0000", 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5509999999999998e-06 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225609+0000", 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046362154000000003 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225650+0000", 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225703+0000", 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3260999999999997e-05 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27763 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9727824179999995, 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046573208999999997, 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27763 2024-05-08T03:20:11.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:11.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179251+0000", 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.102e-06 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179253+0000", 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8229999999999999e-06 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179256+0000", 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4949999999999998e-06 2024-05-08T03:20:11.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225727+0000", 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046471512999999999 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225768+0000", 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0142999999999998e-05 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225822+0000", 2024-05-08T03:20:11.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4926000000000002e-05 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27764 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9727730799999996, 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046778976999999999, 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27764 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179381+0000", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000122717 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179384+0000", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5000000000000002e-06 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179389+0000", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179392+0000", 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9280000000000003e-06 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225880+0000", 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046487354000000002 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225944+0000", 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4657000000000001e-05 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226038+0000", 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3066999999999995e-05 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27765 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 802816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9726315400000001, 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046843500000000003, 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.215 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27765 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181093+0000", 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016927820000000001 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181096+0000", 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.506e-06 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181103+0000", 2024-05-08T03:20:11.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181108+0000", 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5979999999999996e-06 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226080+0000", 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044971481000000001 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226152+0000", 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2057000000000005e-05 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226244+0000", 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1891000000000005e-05 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27776 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:11.219 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9706950000000001, 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046303351999999999, 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27776 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2599999999999998e-07 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181338+0000", 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.266e-06 2024-05-08T03:20:11.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181341+0000", 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181344+0000", 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227544+0000", 2024-05-08T03:20:11.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046199656999999998 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227585+0000", 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1850000000000001e-05 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227640+0000", 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4586000000000002e-05 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.223 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27777 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9706848050000003, 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046409785000000002, 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27777 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181348+0000", 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.145e-06 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181351+0000", 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8210000000000002e-06 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181354+0000", 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5900000000000002e-06 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227662+0000", 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046308716 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227703+0000", 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0141000000000001e-05 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227757+0000", 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4147000000000002e-05 2024-05-08T03:20:11.227 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27778 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9706750839999998, 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046521521000000003, 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27778 2024-05-08T03:20:11.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:11.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181358+0000", 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.167e-06 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181362+0000", 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181364+0000", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6019999999999998e-06 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227780+0000", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046415911999999997 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227821+0000", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1179e-05 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227878+0000", 2024-05-08T03:20:11.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6736e-05 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27779 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9706645500000004, 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046626723000000002, 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27779 2024-05-08T03:20:11.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183095+0000", 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001728282 2024-05-08T03:20:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183102+0000", 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4790000000000001e-06 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183117+0000", 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4929999999999999e-05 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183126+0000", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9209999999999998e-06 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227901+0000", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044775342000000003 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227940+0000", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9332999999999998e-05 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227994+0000", 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3436000000000001e-05 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27794 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1269760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9685258489999997, 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046263348000000003, 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.236 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27794 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183508+0000", 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.057e-06 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183514+0000", 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6710000000000004e-06 2024-05-08T03:20:11.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183519+0000", 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8269999999999997e-06 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229676+0000", 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046157090999999997 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229716+0000", 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9870000000000003e-05 2024-05-08T03:20:11.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229769+0000", 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3477000000000003e-05 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27795 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1294336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9685069850000003, 2024-05-08T03:20:11.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046360547000000002, 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27795 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183527+0000", 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183533+0000", 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183537+0000", 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4660000000000001e-06 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229792+0000", 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046254587999999999 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229832+0000", 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0055999999999997e-05 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229885+0000", 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3488e-05 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27796 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1441792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 6.9684885369999998, 2024-05-08T03:20:11.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046459223000000001, 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27796 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185095+0000", 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015521389999999999 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185101+0000", 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:20:11.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185115+0000", 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3735e-05 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185123+0000", 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1359999999999997e-06 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229910+0000", 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044786908 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229949+0000", 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9192e-05 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.230002+0000", 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3601999999999999e-05 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.248 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:11.250 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:11.250 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-05-08T03:20:11.250 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:11.248+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:11.266 INFO:teuthology.orchestra.run.smithi177.stdout:2324 flatten 2024-05-08T03:20:11.295 INFO:teuthology.orchestra.run.smithi177.stdout:2328 trunc from 0xb5c937 to 0x639aa4c 2024-05-08T03:20:11.301 INFO:teuthology.orchestra.run.smithi177.stdout:2330 write 0x388e287 thru 0x3899c86 (0xba00 bytes) 2024-05-08T03:20:11.304 INFO:teuthology.orchestra.run.smithi177.stdout:2331 write 0x2e3d35b thru 0x2e41ced (0x4993 bytes) 2024-05-08T03:20:11.320 INFO:teuthology.orchestra.run.smithi177.stdout:2332 write 0xc9cc2ba thru 0xc9d3e82 (0x7bc9 bytes) 2024-05-08T03:20:11.323 INFO:teuthology.orchestra.run.smithi177.stdout:2333 punch from 0x4790145 to 0x479e5a8, (0xe463 bytes) 2024-05-08T03:20:11.324 INFO:teuthology.orchestra.run.smithi177.stdout:2335 write 0xac9d80e thru 0xaca0404 (0x2bf7 bytes) 2024-05-08T03:20:11.325 INFO:teuthology.orchestra.run.smithi177.stdout:2336 writesame 0xdd4873 thru 0xdd72d9 data_size 0xd(0x2a67 bytes) 2024-05-08T03:20:11.328 INFO:teuthology.orchestra.run.smithi177.stdout:2337 write 0x65f19fa thru 0x65f444c (0x2a53 bytes) 2024-05-08T03:20:11.329 INFO:teuthology.orchestra.run.smithi177.stdout:2338 write 0xad7efcc thru 0xad845e8 (0x561d bytes) 2024-05-08T03:20:11.332 INFO:teuthology.orchestra.run.smithi177.stdout:2339 writesame 0x53bca3b thru 0x53c0ae7 data_size 0x1(0x40ad bytes) 2024-05-08T03:20:11.340 INFO:teuthology.orchestra.run.smithi177.stdout:2340 writesame 0xb782b13 thru 0xb791b68 data_size 0x1(0xf056 bytes) 2024-05-08T03:20:11.350 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:11.348+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:11.372 INFO:teuthology.orchestra.run.smithi177.stdout:2342 trunc from 0xc9d3e83 to 0xd5a4f76 2024-05-08T03:20:11.377 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.377 INFO:teuthology.orchestra.run.smithi177.stdout:2345 write 0x5caef96 thru 0x5cb9a6b (0xaad6 bytes) 2024-05-08T03:20:11.379 INFO:teuthology.orchestra.run.smithi177.stdout:2346 punch from 0x18f1b1d to 0x18fd2d4, (0xb7b7 bytes) 2024-05-08T03:20:11.381 INFO:teuthology.orchestra.run.smithi177.stdout:2347 read 0x467b635 thru 0x467bcff (0x6cb bytes) 2024-05-08T03:20:11.381 INFO:teuthology.orchestra.run.smithi177.stdout:2348 punch from 0x991525 to 0x993b2c, (0x2607 bytes) 2024-05-08T03:20:11.381 INFO:teuthology.orchestra.run.smithi177.stdout:2350 write 0x954a65a thru 0x954bf48 (0x18ef bytes) 2024-05-08T03:20:11.392 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:11.392 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:11.392 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:11.392 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:11.393 INFO:teuthology.orchestra.run.smithi177.stdout:2351 trunc from 0xd5a4f76 to 0x3141df9 2024-05-08T03:20:11.403 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:11.421 INFO:teuthology.orchestra.run.smithi177.stdout:2353 write 0x54037a8 thru 0x540a406 (0x6c5f bytes) 2024-05-08T03:20:11.424 INFO:teuthology.orchestra.run.smithi177.stdout:2355 read 0x4bc462c thru 0x4bcc03a (0x7a0f bytes) 2024-05-08T03:20:11.424 INFO:teuthology.orchestra.run.smithi177.stdout:2357 read 0x327e7d1 thru 0x328653d (0x7d6d bytes) 2024-05-08T03:20:11.424 INFO:teuthology.orchestra.run.smithi177.stdout:2358 write 0xc0758d thru 0xc0859e (0x1012 bytes) 2024-05-08T03:20:11.425 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.425 INFO:teuthology.orchestra.run.smithi177.stdout:2360 read 0x390a8bd thru 0x391a79a (0xfede bytes) 2024-05-08T03:20:11.438 INFO:teuthology.orchestra.run.smithi177.stdout:2361 write 0xa4abca4 thru 0xa4ae518 (0x2875 bytes) 2024-05-08T03:20:11.444 INFO:teuthology.orchestra.run.smithi177.stdout:2362 trunc from 0xa4ae519 to 0xe9b4a14 2024-05-08T03:20:11.450 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:11.449+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:11.454 INFO:teuthology.orchestra.run.smithi177.stdout:2364 writesame 0xbb3c1e8 thru 0xbb46dd1 data_size 0x87(0xabea bytes) 2024-05-08T03:20:11.456 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.456 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.456 INFO:teuthology.orchestra.run.smithi177.stdout:2368 writesame 0x2abd889 thru 0x2abe539 data_size 0x13(0xcb1 bytes) 2024-05-08T03:20:11.459 INFO:teuthology.orchestra.run.smithi177.stdout:2370 write 0xb5f4873 thru 0xb600efc (0xc68a bytes) 2024-05-08T03:20:11.461 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.461 INFO:teuthology.orchestra.run.smithi177.stdout:2377 writesame 0xe1c6c6d thru 0xe1ce0a6 data_size 0x57(0x743a bytes) 2024-05-08T03:20:11.465 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.465 INFO:teuthology.orchestra.run.smithi177.stdout:2379 write 0x1b19c13 thru 0x1b29ba8 (0xff96 bytes) 2024-05-08T03:20:11.466 INFO:teuthology.orchestra.run.smithi177.stdout:2380 read 0x9e62529 thru 0x9e6323c (0xd14 bytes) 2024-05-08T03:20:11.467 INFO:teuthology.orchestra.run.smithi177.stdout:2381 read 0x3cc60f2 thru 0x3cc99c3 (0x38d2 bytes) 2024-05-08T03:20:11.467 INFO:teuthology.orchestra.run.smithi177.stdout:2383 read 0x950a6f thru 0x9530ee (0x2680 bytes) 2024-05-08T03:20:11.467 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.467 INFO:teuthology.orchestra.run.smithi177.stdout:2387 read 0x66e7e3f thru 0x66e9872 (0x1a34 bytes) 2024-05-08T03:20:11.468 INFO:teuthology.orchestra.run.smithi177.stdout:2388 writesame 0x5a56c10 thru 0x5a61f5f data_size 0x4c(0xb350 bytes) 2024-05-08T03:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout:2389 trunc from 0xe9b4a14 to 0x2776b01 2024-05-08T03:20:11.506 INFO:teuthology.orchestra.run.smithi177.stdout:2391 writesame 0x58f3ef1 thru 0x58f535f data_size 0x1(0x146f bytes) 2024-05-08T03:20:11.508 INFO:teuthology.orchestra.run.smithi177.stdout:2392 write 0x22834d4 thru 0x228393a (0x467 bytes) 2024-05-08T03:20:11.509 INFO:teuthology.orchestra.run.smithi177.stdout:2395 read 0x2ac0228 thru 0x2ac6895 (0x666e bytes) 2024-05-08T03:20:11.509 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.509 INFO:teuthology.orchestra.run.smithi177.stdout:2398 read 0x1ac241a thru 0x1ad037b (0xdf62 bytes) 2024-05-08T03:20:11.510 INFO:teuthology.orchestra.run.smithi177.stdout:2399 write 0x321c9f3 thru 0x3229e0b (0xd419 bytes) 2024-05-08T03:20:11.511 INFO:teuthology.orchestra.run.smithi177.stdout:2402 read 0x2f36f49 thru 0x2f3bc2b (0x4ce3 bytes) 2024-05-08T03:20:11.512 INFO:teuthology.orchestra.run.smithi177.stdout:2403 punch from 0x5759e19 to 0x57640e0, (0xa2c7 bytes) 2024-05-08T03:20:11.512 INFO:teuthology.orchestra.run.smithi177.stdout:2405 write 0xe7d069 thru 0xe867f2 (0x978a bytes) 2024-05-08T03:20:11.513 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.520 INFO:teuthology.orchestra.run.smithi177.stdout:2407 write 0x66aad9c thru 0x66b541a (0xa67f bytes) 2024-05-08T03:20:11.532 INFO:teuthology.orchestra.run.smithi177.stdout:2409 write 0xa4347f5 thru 0xa4373d6 (0x2be2 bytes) 2024-05-08T03:20:11.533 INFO:teuthology.orchestra.run.smithi177.stdout:2410 write 0x85d40e6 thru 0x85dbabc (0x79d7 bytes) 2024-05-08T03:20:11.535 INFO:teuthology.orchestra.run.smithi177.stdout:2411 read 0x116a951 thru 0x116d614 (0x2cc4 bytes) 2024-05-08T03:20:11.535 INFO:teuthology.orchestra.run.smithi177.stdout:2412 writesame 0x5edefac thru 0x5eecda7 data_size 0x2(0xddfc bytes) 2024-05-08T03:20:11.549 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:11.549 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:11.549 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:11.549 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:11.549 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:11.551 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:11.550+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:11.552 INFO:teuthology.orchestra.run.smithi177.stdout:2414 punch from 0x8ad9b02 to 0x8ae1a7d, (0x7f7b bytes) 2024-05-08T03:20:11.554 INFO:teuthology.orchestra.run.smithi177.stdout:2415 trunc from 0xa4373d7 to 0xc67a71f 2024-05-08T03:20:11.559 INFO:teuthology.orchestra.run.smithi177.stdout:2416 read 0x25a8696 thru 0x25b2716 (0xa081 bytes) 2024-05-08T03:20:11.560 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:11.561 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.561 INFO:teuthology.orchestra.run.smithi177.stdout:2419 write 0x584cd1a thru 0x5853c8b (0x6f72 bytes) 2024-05-08T03:20:11.561 INFO:teuthology.orchestra.run.smithi177.stdout:2420 read 0x7885f6f thru 0x788f434 (0x94c6 bytes) 2024-05-08T03:20:11.562 INFO:teuthology.orchestra.run.smithi177.stdout:2421 writesame 0xbdc6512 thru 0xbdce86c data_size 0xb(0x835b bytes) 2024-05-08T03:20:11.564 INFO:teuthology.orchestra.run.smithi177.stdout:2422 writesame 0x51c4e4d thru 0x51cd923 data_size 0x1(0x8ad7 bytes) 2024-05-08T03:20:11.581 INFO:teuthology.orchestra.run.smithi177.stdout:2424 punch from 0x8b84caa to 0x8b8cff8, (0x834e bytes) 2024-05-08T03:20:11.581 INFO:teuthology.orchestra.run.smithi177.stdout:2425 read 0x7b07fb9 thru 0x7b0c38d (0x43d5 bytes) 2024-05-08T03:20:11.582 INFO:teuthology.orchestra.run.smithi177.stdout:2427 writesame 0x6375a05 thru 0x637fe22 data_size 0x7(0xa41e bytes) 2024-05-08T03:20:11.586 INFO:teuthology.orchestra.run.smithi177.stdout:2429 punch from 0xa727c7 to 0xa7d770, (0xafa9 bytes) 2024-05-08T03:20:11.586 INFO:teuthology.orchestra.run.smithi177.stdout:2430 punch from 0x1fc9820 to 0x1fcc221, (0x2a01 bytes) 2024-05-08T03:20:11.590 INFO:teuthology.orchestra.run.smithi177.stdout:2431 trunc from 0xc67a71f to 0x8af5f1c 2024-05-08T03:20:11.598 INFO:teuthology.orchestra.run.smithi177.stdout:2432 writesame 0x259daf thru 0x2649c5 data_size 0x63(0xac17 bytes) 2024-05-08T03:20:11.599 INFO:teuthology.orchestra.run.smithi177.stdout:2433 write 0xfe2411 thru 0xfe8b6b (0x675b bytes) 2024-05-08T03:20:11.600 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.612 INFO:teuthology.orchestra.run.smithi177.stdout:2437 writesame 0xda48333 thru 0xda4cfc0 data_size 0x29(0x4c8e bytes) 2024-05-08T03:20:11.618 INFO:teuthology.orchestra.run.smithi177.stdout:2439 writesame 0xe2538a0 thru 0xe25c341 data_size 0x82(0x8aa2 bytes) 2024-05-08T03:20:11.618 INFO:teuthology.orchestra.run.smithi177.stdout:2440 punch from 0x1c3d6ef to 0x1c4c8ac, (0xf1bd bytes) 2024-05-08T03:20:11.618 INFO:teuthology.orchestra.run.smithi177.stdout:2441 writesame 0xbd3cc50 thru 0xbd446d4 data_size 0x55(0x7a85 bytes) 2024-05-08T03:20:11.622 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.636 INFO:teuthology.orchestra.run.smithi177.stdout:2443 trunc from 0xe25c342 to 0x234b7a4 2024-05-08T03:20:11.652 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:11.651+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:11.662 INFO:teuthology.orchestra.run.smithi177.stdout:2444 write 0xb08e884 thru 0xb0919ce (0x314b bytes) 2024-05-08T03:20:11.669 INFO:teuthology.orchestra.run.smithi177.stdout:2445 writesame 0xb2e546e thru 0xb2f35f9 data_size 0xa(0xe18c bytes) 2024-05-08T03:20:11.671 INFO:teuthology.orchestra.run.smithi177.stdout:2447 punch from 0x7fc82cc to 0x7fcc75c, (0x4490 bytes) 2024-05-08T03:20:11.671 INFO:teuthology.orchestra.run.smithi177.stdout:2450 read 0x96c4cf5 thru 0x96cd203 (0x850f bytes) 2024-05-08T03:20:11.671 INFO:teuthology.orchestra.run.smithi177.stdout:2451 read 0x2aad6da thru 0x2ab6411 (0x8d38 bytes) 2024-05-08T03:20:11.671 INFO:teuthology.orchestra.run.smithi177.stdout:2454 write 0x7fecf4c thru 0x7ff0fd3 (0x4088 bytes) 2024-05-08T03:20:11.672 INFO:teuthology.orchestra.run.smithi177.stdout:2455 punch from 0x25e0176 to 0x25e25ee, (0x2478 bytes) 2024-05-08T03:20:11.672 INFO:teuthology.orchestra.run.smithi177.stdout:2457 read 0x819abb3 thru 0x81a47ec (0x9c3a bytes) 2024-05-08T03:20:11.672 INFO:teuthology.orchestra.run.smithi177.stdout:2458 read 0x6b1de98 thru 0x6b2ad46 (0xceaf bytes) 2024-05-08T03:20:11.673 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.673 INFO:teuthology.orchestra.run.smithi177.stdout:2461 punch from 0xa9c9ec8 to 0xa9d1f2c, (0x8064 bytes) 2024-05-08T03:20:11.673 INFO:teuthology.orchestra.run.smithi177.stdout:2462 writesame 0x5943346 thru 0x5948dc6 data_size 0x1(0x5a81 bytes) 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:11.684 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5819635249999999, 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:11.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:11.688 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.580243055, 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.689 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:11.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5802096140000002, 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:11.693 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:11.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.697 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5787838750000001, 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.698 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:11.702 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5786912609999999, 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.706 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5781805869999999, 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.707 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:11.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:11.711 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5781530789999998, 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:11.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:11.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:11.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5777466479999998, 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:11.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:11.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:11.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:11.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5772912699999999, 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:11.720 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.724 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5767630229999998, 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.725 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.729 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5762905250000001, 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:11.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:11.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.576212393, 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.734 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:11.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:11.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.738 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5761369709999999, 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:11.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:11.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:11.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5760598080000001, 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.743 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:11.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:11.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5759822850000003, 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.747 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.751 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5759553730000002, 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:11.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:11.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5759341629999999, 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.756 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:11.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:11.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5759143829999998, 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.760 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:11.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:11.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:11.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:11.764 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.575359857, 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:11.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:11.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:11.768 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.5753408449999999, 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.769 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:11.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:11.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:11.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:11.773 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:11.774 INFO:teuthology.orchestra.run.smithi177.stdout:2464 trunc from 0xb2f35fa to 0xcb82f9a 2024-05-08T03:20:11.774 INFO:teuthology.orchestra.run.smithi177.stdout:2466 read 0x6f83ebf thru 0x6f895a8 (0x56ea bytes) 2024-05-08T03:20:11.774 INFO:teuthology.orchestra.run.smithi177.stdout:2467 writesame 0x340c288 thru 0x341b3af data_size 0x4(0xf128 bytes) 2024-05-08T03:20:11.775 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2469 punch from 0xbfe15e6 to 0xbfe9fbb, (0x89d5 bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2470 trunc from 0xcb82f9a to 0x51b3b3e 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2472 write 0x654b25e thru 0x655a70a (0xf4ad bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2473 read 0x150f685 thru 0x151bb89 (0xc505 bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2474 read 0x55b5d5c thru 0x55c405a (0xe2ff bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2476 trunc from 0x655a70b to 0x1ba4b72 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2477 write 0x719eb07 thru 0x71a2a4c (0x3f46 bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2478 writesame 0x96b0bb thru 0x975686 data_size 0x51(0xa5cc bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2479 writesame 0x844ec72 thru 0x845a3d9 data_size 0x4(0xb768 bytes) 2024-05-08T03:20:11.777 INFO:teuthology.orchestra.run.smithi177.stdout:2480 punch from 0xfbd0f5 to 0xfc631f, (0x922a bytes) 2024-05-08T03:20:11.778 INFO:teuthology.orchestra.run.smithi177.stdout:2482 write 0xb001de5 thru 0xb00f4ee (0xd70a bytes) 2024-05-08T03:20:11.778 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:11.772+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:11.787 INFO:teuthology.orchestra.run.smithi177.stdout:2483 trunc from 0xb00f4ef to 0x483a87 2024-05-08T03:20:11.794 INFO:teuthology.orchestra.run.smithi177.stdout:2484 punch from 0x34be56 to 0x352409, (0x65b3 bytes) 2024-05-08T03:20:11.794 INFO:teuthology.orchestra.run.smithi177.stdout:2487 read 0x1cc656 thru 0x1cd646 (0xff1 bytes) 2024-05-08T03:20:11.802 INFO:teuthology.orchestra.run.smithi177.stdout:2488 writesame 0x3113045 thru 0x3113f60 data_size 0x2(0xf1c bytes) 2024-05-08T03:20:11.815 INFO:teuthology.orchestra.run.smithi177.stdout:2489 write 0x7cb3e68 thru 0x7cbbc08 (0x7da1 bytes) 2024-05-08T03:20:11.822 INFO:teuthology.orchestra.run.smithi177.stdout:2490 write 0x883dfd1 thru 0x88474db (0x950b bytes) 2024-05-08T03:20:11.827 INFO:teuthology.orchestra.run.smithi177.stdout:2491 writesame 0x4003da2 thru 0x400cfa9 data_size 0x4(0x9208 bytes) 2024-05-08T03:20:11.832 INFO:teuthology.orchestra.run.smithi177.stdout:2492 punch from 0x6e78198 to 0x6e79ea6, (0x1d0e bytes) 2024-05-08T03:20:11.832 INFO:teuthology.orchestra.run.smithi177.stdout:2493 writesame 0x770c09c thru 0x7714dbf data_size 0x6(0x8d24 bytes) 2024-05-08T03:20:11.844 INFO:teuthology.orchestra.run.smithi177.stdout:2494 writesame 0xb650c6b thru 0xb654530 data_size 0x2b(0x38c6 bytes) 2024-05-08T03:20:11.845 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.845 INFO:teuthology.orchestra.run.smithi177.stdout:2497 punch from 0x601e24e to 0x601fe6d, (0x1c1f bytes) 2024-05-08T03:20:11.845 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.845 INFO:teuthology.orchestra.run.smithi177.stdout:2499 write 0x47c5e6 thru 0x48453d (0x7f58 bytes) 2024-05-08T03:20:11.852 INFO:teuthology.orchestra.run.smithi177.stdout:2500 trunc from 0xb654531 to 0xdbf674d 2024-05-08T03:20:11.857 INFO:teuthology.orchestra.run.smithi177.stdout:2501 punch from 0x68bf033 to 0x68c50a8, (0x6075 bytes) 2024-05-08T03:20:11.857 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.857 INFO:teuthology.orchestra.run.smithi177.stdout:2503 read 0xd1c87f1 thru 0xd1cf058 (0x6868 bytes) 2024-05-08T03:20:11.857 INFO:teuthology.orchestra.run.smithi177.stdout:2504 read 0x87f4bd9 thru 0x8803ce1 (0xf109 bytes) 2024-05-08T03:20:11.862 INFO:teuthology.orchestra.run.smithi177.stdout:2505 trunc from 0xdbf674d to 0xa993119 2024-05-08T03:20:11.868 INFO:teuthology.orchestra.run.smithi177.stdout:2506 punch from 0x2198ccf to 0x21a3118, (0xa449 bytes) 2024-05-08T03:20:11.869 INFO:teuthology.orchestra.run.smithi177.stdout:2507 trunc from 0xa993119 to 0xad48fe0 2024-05-08T03:20:11.874 INFO:teuthology.orchestra.run.smithi177.stdout:2509 write 0x709530d thru 0x709fbd3 (0xa8c7 bytes) 2024-05-08T03:20:11.874 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:11.873+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:11.885 INFO:teuthology.orchestra.run.smithi177.stdout:2510 trunc from 0xad48fe0 to 0x285c88c 2024-05-08T03:20:11.892 INFO:teuthology.orchestra.run.smithi177.stdout:2512 read 0x16dbd84 thru 0x16e4cb4 (0x8f31 bytes) 2024-05-08T03:20:11.897 INFO:teuthology.orchestra.run.smithi177.stdout:2513 write 0x2b88208 thru 0x2b8e917 (0x6710 bytes) 2024-05-08T03:20:11.920 INFO:teuthology.orchestra.run.smithi177.stdout:2514 write 0xdb8471b thru 0xdb9448e (0xfd74 bytes) 2024-05-08T03:20:11.923 INFO:teuthology.orchestra.run.smithi177.stdout:2515 punch from 0x31416df to 0x3145c4e, (0x456f bytes) 2024-05-08T03:20:11.923 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:11.923 INFO:teuthology.orchestra.run.smithi177.stdout:2517 write 0x652746 thru 0x658c07 (0x64c2 bytes) 2024-05-08T03:20:11.927 INFO:teuthology.orchestra.run.smithi177.stdout:2518 write 0x192606e thru 0x192ca45 (0x69d8 bytes) 2024-05-08T03:20:11.927 INFO:teuthology.orchestra.run.smithi177.stdout:2519 read 0x45f9436 thru 0x4600ebb (0x7a86 bytes) 2024-05-08T03:20:11.927 INFO:teuthology.orchestra.run.smithi177.stdout:2520 read 0xc31067a thru 0xc31eb65 (0xe4ec bytes) 2024-05-08T03:20:11.927 INFO:teuthology.orchestra.run.smithi177.stdout:2521 punch from 0x5e38da7 to 0x5e44f78, (0xc1d1 bytes) 2024-05-08T03:20:11.928 INFO:teuthology.orchestra.run.smithi177.stdout:2522 punch from 0x854ac01 to 0x855962f, (0xea2e bytes) 2024-05-08T03:20:11.928 INFO:teuthology.orchestra.run.smithi177.stdout:2523 write 0x1978ac2 thru 0x197cc06 (0x4145 bytes) 2024-05-08T03:20:11.928 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:11.928 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:11.928 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:11.928 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:11.945 INFO:teuthology.orchestra.run.smithi177.stdout:2525 trunc from 0xdb9448f to 0x23ab7d7 2024-05-08T03:20:11.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:11.957 INFO:teuthology.orchestra.run.smithi177.stdout:2526 writesame 0xb6a8bd thru 0xb773f2 data_size 0x26(0xcb36 bytes) 2024-05-08T03:20:11.959 INFO:teuthology.orchestra.run.smithi177.stdout:2527 read 0x1f97656 thru 0x1f9d47c (0x5e27 bytes) 2024-05-08T03:20:11.974 INFO:teuthology.orchestra.run.smithi177.stdout:2528 writesame 0xaa79f02 thru 0xaa84d5f data_size 0x2(0xae5e bytes) 2024-05-08T03:20:11.976 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:11.974+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:11.983 INFO:teuthology.orchestra.run.smithi177.stdout:2529 write 0xd159948 thru 0xd15e203 (0x48bc bytes) 2024-05-08T03:20:11.987 INFO:teuthology.orchestra.run.smithi177.stdout:2531 read 0x98bf938 thru 0x98cc8da (0xcfa3 bytes) 2024-05-08T03:20:11.987 INFO:teuthology.orchestra.run.smithi177.stdout:2532 read 0x78c6088 thru 0x78cb57c (0x54f5 bytes) 2024-05-08T03:20:11.987 INFO:teuthology.orchestra.run.smithi177.stdout:2534 write 0x4d5e461 thru 0x4d65ce2 (0x7882 bytes) 2024-05-08T03:20:11.989 INFO:teuthology.orchestra.run.smithi177.stdout:2535 punch from 0x275d43e to 0x2761c3c, (0x47fe bytes) 2024-05-08T03:20:11.989 INFO:teuthology.orchestra.run.smithi177.stdout:2536 write 0x4e0c1fa thru 0x4e12bfc (0x6a03 bytes) 2024-05-08T03:20:12.004 INFO:teuthology.orchestra.run.smithi177.stdout:2539 trunc from 0xd15e204 to 0x1d98d55 2024-05-08T03:20:12.018 INFO:teuthology.orchestra.run.smithi177.stdout:2540 write 0x123a41c thru 0x123f65a (0x523f bytes) 2024-05-08T03:20:12.020 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.025 INFO:teuthology.orchestra.run.smithi177.stdout:2542 write 0x240cdb5 thru 0x241175b (0x49a7 bytes) 2024-05-08T03:20:12.028 INFO:teuthology.orchestra.run.smithi177.stdout:2543 trunc from 0x241175c to 0x1165a13 2024-05-08T03:20:12.036 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.036 INFO:teuthology.orchestra.run.smithi177.stdout:2545 punch from 0x413e28 to 0x41fc2f, (0xbe07 bytes) 2024-05-08T03:20:12.036 INFO:teuthology.orchestra.run.smithi177.stdout:2546 read 0x8276fe thru 0x82dc2e (0x6531 bytes) 2024-05-08T03:20:12.036 INFO:teuthology.orchestra.run.smithi177.stdout:2547 read 0x11825a thru 0x121781 (0x9528 bytes) 2024-05-08T03:20:12.049 INFO:teuthology.orchestra.run.smithi177.stdout:2549 write 0x707a0cc thru 0x707ae74 (0xda9 bytes) 2024-05-08T03:20:12.050 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.062 INFO:teuthology.orchestra.run.smithi177.stdout:2552 write 0xc204fe1 thru 0xc20dae2 (0x8b02 bytes) 2024-05-08T03:20:12.066 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.077 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:12.075+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:12.081 INFO:teuthology.orchestra.run.smithi177.stdout:2554 trunc from 0xc20dae3 to 0x7b3d91 2024-05-08T03:20:12.087 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:12.087 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:12.087 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:12.087 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:12.087 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:12.097 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:12.104 INFO:teuthology.orchestra.run.smithi177.stdout:2556 write 0x9cc483 thru 0x9cc51f (0x9d bytes) 2024-05-08T03:20:12.106 INFO:teuthology.orchestra.run.smithi177.stdout:2557 read 0x8d5ea2 thru 0x8e4eb0 (0xf00f bytes) 2024-05-08T03:20:12.106 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.117 INFO:teuthology.orchestra.run.smithi177.stdout:2560 write 0x441026b thru 0x441b0e3 (0xae79 bytes) 2024-05-08T03:20:12.118 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.136 INFO:teuthology.orchestra.run.smithi177.stdout:2562 write 0xda7b5f6 thru 0xda7c00d (0xa18 bytes) 2024-05-08T03:20:12.138 INFO:teuthology.orchestra.run.smithi177.stdout:2564 read 0xd78297b thru 0xd7892bd (0x6943 bytes) 2024-05-08T03:20:12.149 INFO:teuthology.orchestra.run.smithi177.stdout:2565 trunc from 0xda7c00e to 0x4782319 2024-05-08T03:20:12.160 INFO:teuthology.orchestra.run.smithi177.stdout:2566 read 0x3e58124 thru 0x3e5db11 (0x59ee bytes) 2024-05-08T03:20:12.160 INFO:teuthology.orchestra.run.smithi177.stdout:2567 punch from 0x38da877 to 0x38e011a, (0x58a3 bytes) 2024-05-08T03:20:12.161 INFO:teuthology.orchestra.run.smithi177.stdout:2569 punch from 0xe916e4 to 0xe9b68e, (0x9faa bytes) 2024-05-08T03:20:12.171 INFO:teuthology.orchestra.run.smithi177.stdout:2571 writesame 0x85cf09c thru 0x85db8cd data_size 0x29(0xc832 bytes) 2024-05-08T03:20:12.178 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:12.176+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:12.183 INFO:teuthology.orchestra.run.smithi177.stdout:2572 writesame 0xd2860b0 thru 0xd293f48 data_size 0x83(0xde99 bytes) 2024-05-08T03:20:12.183 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.183 INFO:teuthology.orchestra.run.smithi177.stdout:2574 read 0xbfd53d0 thru 0xbfda39e (0x4fcf bytes) 2024-05-08T03:20:12.184 INFO:teuthology.orchestra.run.smithi177.stdout:2575 writesame 0xa44467e thru 0xa446f1a data_size 0x1(0x289d bytes) 2024-05-08T03:20:12.189 INFO:teuthology.orchestra.run.smithi177.stdout:2576 punch from 0x45f6e2d to 0x4601557, (0xa72a bytes) 2024-05-08T03:20:12.189 INFO:teuthology.orchestra.run.smithi177.stdout:2579 read 0x1b2b4a6 thru 0x1b34165 (0x8cc0 bytes) 2024-05-08T03:20:12.189 INFO:teuthology.orchestra.run.smithi177.stdout:2580 punch from 0x5f68a2c to 0x5f70ca5, (0x8279 bytes) 2024-05-08T03:20:12.189 INFO:teuthology.orchestra.run.smithi177.stdout:2581 write 0x4923fc1 thru 0x4931012 (0xd052 bytes) 2024-05-08T03:20:12.191 INFO:teuthology.orchestra.run.smithi177.stdout:2582 write 0xcb3f2e1 thru 0xcb4eb79 (0xf899 bytes) 2024-05-08T03:20:12.196 INFO:teuthology.orchestra.run.smithi177.stdout:2583 write 0xb6e2669 thru 0xb6f09d9 (0xe371 bytes) 2024-05-08T03:20:12.198 INFO:teuthology.orchestra.run.smithi177.stdout:2584 write 0x289446d thru 0x28a0b53 (0xc6e7 bytes) 2024-05-08T03:20:12.199 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.199 INFO:teuthology.orchestra.run.smithi177.stdout:2588 writesame 0x7786bde thru 0x778bc86 data_size 0x1(0x50a9 bytes) 2024-05-08T03:20:12.209 INFO:teuthology.orchestra.run.smithi177.stdout:2590 read 0x33c3b18 thru 0x33cab00 (0x6fe9 bytes) 2024-05-08T03:20:12.210 INFO:teuthology.orchestra.run.smithi177.stdout:2592 punch from 0x67d98e2 to 0x67e4ece, (0xb5ec bytes) 2024-05-08T03:20:12.210 INFO:teuthology.orchestra.run.smithi177.stdout:2593 punch from 0xb2f72a to 0xb343af, (0x4c85 bytes) 2024-05-08T03:20:12.210 INFO:teuthology.orchestra.run.smithi177.stdout:2594 trunc from 0xd293f49 to 0xd0a847f 2024-05-08T03:20:12.216 INFO:teuthology.orchestra.run.smithi177.stdout:2595 read 0x647ba63 thru 0x647ca33 (0xfd1 bytes) 2024-05-08T03:20:12.225 INFO:teuthology.orchestra.run.smithi177.stdout:2596 writesame 0xebc8a1d thru 0xebcbec6 data_size 0x3f(0x34aa bytes) 2024-05-08T03:20:12.225 INFO:teuthology.orchestra.run.smithi177.stdout:2597 punch from 0xd7243e8 to 0xd725e78, (0x1a90 bytes) 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.231113544999999, 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.230 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:12.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.235 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.231046081000001, 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.236 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:12.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:12.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.171211748999999, 2024-05-08T03:20:12.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:12.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:12.248 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.104703735999999, 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:12.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:12.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:12.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.07816609, 2024-05-08T03:20:12.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:12.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:12.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:12.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.050254322000001, 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:12.260 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:12.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.264 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23281 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'92, mlcod=53'88)", 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 12.000708803, 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074279843999999998, 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226785+0000", 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.299e-06 2024-05-08T03:20:12.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226794+0000", 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5040000000000008e-06 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226810+0000", 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5781e-05 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226820+0000", 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9310000000000003e-06 2024-05-08T03:20:12.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262249+0000", 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.035428501000000001 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262269+0000", 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0077000000000001e-05 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301028+0000", 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.038759184000000002 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301064+0000", 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5567e-05 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26865 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4288512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3330959369999995, 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041924771999999999, 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.271 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26865 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894397+0000", 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894399+0000", 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6309999999999999e-06 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894403+0000", 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8559999999999998e-06 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894408+0000", 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9259999999999999e-06 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936202+0000", 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041793779000000003 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936255+0000", 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3072000000000003e-05 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936321+0000", 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6128000000000001e-05 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26866 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4460544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.275 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3330795000000002, 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042116690999999998, 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26866 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:12.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894413+0000", 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0400000000000002e-07 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894415+0000", 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5990000000000001e-06 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894420+0000", 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9119999999999997e-06 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894424+0000", 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7299999999999999e-06 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936364+0000", 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041939999999999998 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936447+0000", 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3293000000000004e-05 2024-05-08T03:20:12.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936530+0000", 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2853000000000001e-05 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26867 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4509696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3330647029999998, 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042262936000000001, 2024-05-08T03:20:12.280 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26867 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.894428+0000", 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896093+0000", 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016655789999999999 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896098+0000", 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8180000000000003e-06 2024-05-08T03:20:12.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896109+0000", 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1109e-05 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896118+0000", 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8489999999999995e-06 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936561+0000", 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040442845999999998 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936610+0000", 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9286000000000002e-05 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.936691+0000", 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0449000000000002e-05 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26874 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4386816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3312378549999995, 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041925208999999998, 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26874 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.285 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896255+0000", 2024-05-08T03:20:12.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896258+0000", 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9979999999999999e-06 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896261+0000", 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1470000000000001e-06 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896265+0000", 2024-05-08T03:20:12.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9419999999999997e-06 2024-05-08T03:20:12.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938052+0000", 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041787283000000001 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938114+0000", 2024-05-08T03:20:12.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2020000000000006e-05 2024-05-08T03:20:12.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938180+0000", 2024-05-08T03:20:12.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5611999999999995e-05 2024-05-08T03:20:12.290 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26875 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4411392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3312224930000003, 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.04205014, 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.291 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26875 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896270+0000", 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9200000000000002e-07 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896272+0000", 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6789999999999999e-06 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896277+0000", 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8289999999999998e-06 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896281+0000", 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.839e-06 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938214+0000", 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041933639000000002 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938260+0000", 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5510000000000003e-05 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938320+0000", 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0251999999999998e-05 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26876 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4435968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3312059729999994, 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042168543000000003, 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26876 2024-05-08T03:20:12.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896287+0000", 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5400000000000002e-07 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896289+0000", 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9470000000000002e-06 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896292+0000", 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5470000000000001e-06 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896296+0000", 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.574e-06 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938354+0000", 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042057861000000002 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938398+0000", 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4221999999999998e-05 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938455+0000", 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7037999999999998e-05 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26877 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4608000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.300 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3311930420000007, 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042279939000000002, 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26877 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:12.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896299+0000", 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896300+0000", 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896301+0000", 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.159e-06 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896304+0000", 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.881e-06 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896307+0000", 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:12.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938478+0000", 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042171023000000002 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938524+0000", 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6109999999999997e-05 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938579+0000", 2024-05-08T03:20:12.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5448000000000002e-05 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26878 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4632576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3311825329999998, 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042399394, 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26878 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.896310+0000", 2024-05-08T03:20:12.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898092+0000", 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0017816349999999999 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898096+0000", 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7720000000000002e-06 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898110+0000", 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3764000000000001e-05 2024-05-08T03:20:12.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898118+0000", 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1659999999999994e-06 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938611+0000", 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040492570999999998 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938654+0000", 2024-05-08T03:20:12.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3133000000000002e-05 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.938709+0000", 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5352999999999997e-05 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26892 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4853760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3290361710000003, 2024-05-08T03:20:12.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041959164, 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26892 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898456+0000", 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898457+0000", 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3399999999999999e-07 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898459+0000", 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2960000000000001e-06 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898466+0000", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2400000000000001e-06 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898471+0000", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.532e-06 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940319+0000", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041847652999999999 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940360+0000", 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0952999999999999e-05 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940415+0000", 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5955999999999998e-05 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26893 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4878336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3290157219999994, 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042048864999999998, 2024-05-08T03:20:12.313 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26893 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898477+0000", 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0600000000000001e-07 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898479+0000", 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898489+0000", 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6900000000000004e-06 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898493+0000", 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5709999999999999e-06 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940438+0000", 2024-05-08T03:20:12.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041944253000000001 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940475+0000", 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7512000000000003e-05 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940526+0000", 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0599000000000001e-05 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26894 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3289906570000003, 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042154042000000003, 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26894 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.898502+0000", 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900091+0000", 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015893400000000001 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900096+0000", 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900110+0000", 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3773e-05 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900119+0000", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3880000000000006e-06 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940559+0000", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040439875 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940601+0000", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1950999999999999e-05 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.940656+0000", 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4960999999999998e-05 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26909 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.327027266, 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041961032000000002, 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.322 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26909 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900465+0000", 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900466+0000", 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3799999999999998e-07 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900468+0000", 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.447e-06 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900475+0000", 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9380000000000003e-06 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900479+0000", 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2030000000000002e-06 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942337+0000", 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041857710999999999 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942376+0000", 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9533999999999998e-05 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942426+0000", 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9861000000000003e-05 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:26910 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:13 [read 5566464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.3270072670000008, 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.042062763000000003, 2024-05-08T03:20:12.326 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 26910 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.900485+0000", 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902091+0000", 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001605778 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902096+0000", 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5759999999999999e-06 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902108+0000", 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2018000000000001e-05 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.902116+0000", 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8429999999999999e-06 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942456+0000", 2024-05-08T03:20:12.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.040339331999999999 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942496+0000", 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9774999999999998e-05 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.942548+0000", 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2441000000000002e-05 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.330 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.331 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:12.331 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:12.331 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:12.331 INFO:teuthology.orchestra.run.smithi177.stdout:2598 write 0xed38d8a thru 0xed3cc9e (0x3f15 bytes) 2024-05-08T03:20:12.331 INFO:teuthology.orchestra.run.smithi177.stdout:2599 writesame 0xa7d20f8 thru 0xa7d34c0 data_size 0x1(0x13c9 bytes) 2024-05-08T03:20:12.331 INFO:teuthology.orchestra.run.smithi177.stdout:2601 write 0xc79b141 thru 0xc7a72dd (0xc19d bytes) 2024-05-08T03:20:12.332 INFO:teuthology.orchestra.run.smithi177.stdout:2602 read 0x56f2488 thru 0x56f8f22 (0x6a9b bytes) 2024-05-08T03:20:12.333 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2603 trunc from 0xed3cc9f to 0xa232e8b 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2606 punch from 0xac24ed to 0xac8455, (0x5f68 bytes) 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2607 punch from 0x953128c to 0x953339a, (0x210e bytes) 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2608 punch from 0x7f0a015 to 0x7f150cd, (0xb0b8 bytes) 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2610 punch from 0x934c4e8 to 0x9353e16, (0x792e bytes) 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2611 read 0x16511fa thru 0x1653280 (0x2087 bytes) 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2615 writesame 0xb6bbac5 thru 0xb6c66dc data_size 0x4(0xac18 bytes) 2024-05-08T03:20:12.336 INFO:teuthology.orchestra.run.smithi177.stdout:2616 punch from 0x214aa8d to 0x214d4de, (0x2a51 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2617 write 0xd79f7af thru 0xd7abbe9 (0xc43b bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2618 write 0xa1d77a0 thru 0xa1d86d4 (0xf35 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2619 punch from 0x71acf5d to 0x71b7bdf, (0xac82 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2620 trunc from 0xd7abbea to 0xa7e86d 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2621 read 0x234809 thru 0x23e878 (0xa070 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2622 read 0x3c8488 thru 0x3cf5e0 (0x7159 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2623 punch from 0x8ce71 to 0x932ce, (0x645d bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2625 punch from 0x352757 to 0x356ec1, (0x476a bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2626 write 0x53a550e thru 0x53a93fe (0x3ef1 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2627 writesame 0x1cfd7a8 thru 0x1cfd8c2 data_size 0x1(0x11b bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2629 write 0x8c94f4c thru 0x8ca097c (0xba31 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2634 read 0x1322c88 thru 0x1329534 (0x68ad bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2635 read 0x5e95c25 thru 0x5e9a04e (0x442a bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2639 read 0x7fc35d thru 0x80195f (0x5603 bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2640 write 0x61c3470 thru 0x61c623a (0x2dcb bytes) 2024-05-08T03:20:12.337 INFO:teuthology.orchestra.run.smithi177.stdout:2641 read 0x337365f thru 0x337c175 (0x8b17 bytes) 2024-05-08T03:20:12.338 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:12.333+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:12.343 INFO:teuthology.orchestra.run.smithi177.stdout:2643 writesame 0xbe29d51 thru 0xbe2e0d7 data_size 0x1(0x4387 bytes) 2024-05-08T03:20:12.344 INFO:teuthology.orchestra.run.smithi177.stdout:2644 write 0x4090e80 thru 0x4097ed8 (0x7059 bytes) 2024-05-08T03:20:12.348 INFO:teuthology.orchestra.run.smithi177.stdout:2645 writesame 0xa71f973 thru 0xa72ee7f data_size 0xb(0xf50d bytes) 2024-05-08T03:20:12.352 INFO:teuthology.orchestra.run.smithi177.stdout:2646 writesame 0x990a284 thru 0x9918c08 data_size 0x1(0xe985 bytes) 2024-05-08T03:20:12.382 INFO:teuthology.orchestra.run.smithi177.stdout:2648 trunc from 0xbe2e0d8 to 0xa9f1eb7 2024-05-08T03:20:12.390 INFO:teuthology.orchestra.run.smithi177.stdout:2649 write 0x504ce76 thru 0x5055bba (0x8d45 bytes) 2024-05-08T03:20:12.400 INFO:teuthology.orchestra.run.smithi177.stdout:2651 trunc from 0xa9f1eb7 to 0x34f0098 2024-05-08T03:20:12.419 INFO:teuthology.orchestra.run.smithi177.stdout:2652 trunc from 0x34f0098 to 0x7c8c662 2024-05-08T03:20:12.426 INFO:teuthology.orchestra.run.smithi177.stdout:2653 punch from 0x73abd55 to 0x73bb58a, (0xf835 bytes) 2024-05-08T03:20:12.426 INFO:teuthology.orchestra.run.smithi177.stdout:2654 writesame 0xa880a3 thru 0xa965c3 data_size 0x1(0xe521 bytes) 2024-05-08T03:20:12.435 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:12.434+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:12.442 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:12.442 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:12.442 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:12.442 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:12.451 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:12.454 INFO:teuthology.orchestra.run.smithi177.stdout:2658 writesame 0x2a5ce3 thru 0x2a8658 data_size 0x3d(0x2976 bytes) 2024-05-08T03:20:12.455 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.455 INFO:teuthology.orchestra.run.smithi177.stdout:2661 punch from 0x2ec286a to 0x2ec70e7, (0x487d bytes) 2024-05-08T03:20:12.455 INFO:teuthology.orchestra.run.smithi177.stdout:2662 read 0x232af11 thru 0x2332f06 (0x7ff6 bytes) 2024-05-08T03:20:12.456 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.461 INFO:teuthology.orchestra.run.smithi177.stdout:2665 write 0x873efad thru 0x874349a (0x44ee bytes) 2024-05-08T03:20:12.466 INFO:teuthology.orchestra.run.smithi177.stdout:2666 read 0x268dd24 thru 0x2699066 (0xb343 bytes) 2024-05-08T03:20:12.466 INFO:teuthology.orchestra.run.smithi177.stdout:2667 write 0x1a24847 thru 0x1a2c539 (0x7cf3 bytes) 2024-05-08T03:20:12.468 INFO:teuthology.orchestra.run.smithi177.stdout:2671 read 0x2bde45a thru 0x2be8120 (0x9cc7 bytes) 2024-05-08T03:20:12.475 INFO:teuthology.orchestra.run.smithi177.stdout:2674 trunc from 0x874349b to 0x308e5f4 2024-05-08T03:20:12.483 INFO:teuthology.orchestra.run.smithi177.stdout:2675 read 0x811ddc thru 0x818f9b (0x71c0 bytes) 2024-05-08T03:20:12.494 INFO:teuthology.orchestra.run.smithi177.stdout:2676 write 0x7ba2fdd thru 0x7bb0394 (0xd3b8 bytes) 2024-05-08T03:20:12.508 INFO:teuthology.orchestra.run.smithi177.stdout:2680 writesame 0xd901787 thru 0xd905809 data_size 0xf(0x4083 bytes) 2024-05-08T03:20:12.508 INFO:teuthology.orchestra.run.smithi177.stdout:2681 write 0x7c57a89 thru 0x7c5da0c (0x5f84 bytes) 2024-05-08T03:20:12.509 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.509 INFO:teuthology.orchestra.run.smithi177.stdout:2684 write 0x1404160 thru 0x1413ed6 (0xfd77 bytes) 2024-05-08T03:20:12.512 INFO:teuthology.orchestra.run.smithi177.stdout:2686 read 0x94aff6d thru 0x94b10fd (0x1191 bytes) 2024-05-08T03:20:12.512 INFO:teuthology.orchestra.run.smithi177.stdout:2687 punch from 0x2b773c1 to 0x2b873b6, (0xfff5 bytes) 2024-05-08T03:20:12.512 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:12.521 INFO:teuthology.orchestra.run.smithi177.stdout:2691 trunc from 0xd90580a to 0x5e2bba3 2024-05-08T03:20:12.532 INFO:teuthology.orchestra.run.smithi177.stdout:2692 clone 23 order 20 su 8192 sc 8 2024-05-08T03:20:12.537 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:12.535+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:12.557 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:12.557 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:12.557 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:12.557 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:12.558 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:12.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:12.637 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:12.635+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:12.673 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.803573004999997, 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.674 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:20:12.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:20:12.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:20:12.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.803230827, 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:20:12.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:20:12.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:20:12.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.803191994999999, 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.685 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:20:12.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:20:12.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:12.691 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.803145962000002, 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:20:12.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.802128770000003, 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.697 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:12.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:20:12.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:20:12.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.802092518999999, 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.802065220000003, 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:20:12.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:20:12.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.713 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.802035527000001, 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:12.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:20:12.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:20:12.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:20:12.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.802010011999997, 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:20:12.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:20:12.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.724 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.791895846000003, 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:20:12.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:12.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:12.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.730 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539557000999999, 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:12.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:20:12.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:20:12.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539538669999999, 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:20:12.738 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539514209000004, 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.739 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:20:12.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539487866000002, 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.743 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:12.747 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539470475000002, 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:20:12.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:20:12.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539453194000004, 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.755 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539425756999997, 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.756 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.759 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539407797999999, 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539390711999999, 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:20:12.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:20:12.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:12.767 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 44.539373728000001, 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:20:12.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:12.771 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:12.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:12.773 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:12.771+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:12.874 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:12.872+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:12.888 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:12.905 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:12.906 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:12.920 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:12.920 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:12.922 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:12.923 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:12.926 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:12.926 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:12.929 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:13.014 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:13.012+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:13.075 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:13.074+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:13.076 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:13.076 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:13.076 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:13.076 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:13.076 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:13.083 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:20:13.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:13.176 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:13.175+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:13.202 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-05-08T03:20:13.203 INFO:teuthology.orchestra.run.smithi177.stdout:2693 read 0x343df10 thru 0x34461c6 (0x82b7 bytes) 2024-05-08T03:20:13.205 INFO:teuthology.orchestra.run.smithi177.stdout:2694 read 0x2449d30 thru 0x2450ad7 (0x6da8 bytes) 2024-05-08T03:20:13.206 INFO:teuthology.orchestra.run.smithi177.stdout:2695 punch from 0x4045f3a to 0x4048993, (0x2a59 bytes) 2024-05-08T03:20:13.209 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.209 INFO:teuthology.orchestra.run.smithi177.stdout:2697 read 0x3edb4d4 thru 0x3ee47ec (0x9319 bytes) 2024-05-08T03:20:13.210 INFO:teuthology.orchestra.run.smithi177.stdout:2698 read 0x1aa1ba1 thru 0x1aa4e83 (0x32e3 bytes) 2024-05-08T03:20:13.211 INFO:teuthology.orchestra.run.smithi177.stdout:2699 punch from 0x43cb621 to 0x43da233, (0xec12 bytes) 2024-05-08T03:20:13.224 INFO:teuthology.orchestra.run.smithi177.stdout:2702 write 0xc9ad79e thru 0xc9b2b36 (0x5399 bytes) 2024-05-08T03:20:13.230 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.230 INFO:teuthology.orchestra.run.smithi177.stdout:2704 punch from 0x20718a9 to 0x20814bc, (0xfc13 bytes) 2024-05-08T03:20:13.235 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.024933603999997, 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:20:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:20:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:20:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105253087000001, 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:20:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.245 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:20:13.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:20:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.250 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.250 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:13.250 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105229016000003, 2024-05-08T03:20:13.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:20:13.250 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.250 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.251 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.251 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.251 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.251 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.251 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:20:13.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.255 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105203293000002, 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:20:13.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:13.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105171575999996, 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.260 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:13.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:20:13.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105146472000001, 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.265 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:20:13.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.269 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105121666999999, 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.270 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:20:13.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:20:13.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:20:13.275 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105093123000003, 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.276 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:13.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:20:13.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:20:13.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105070601000001, 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:20:13.281 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:13.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:20:13.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:20:13.285 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105040260999999, 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.286 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:20:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:20:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.105019079999998, 2024-05-08T03:20:13.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:20:13.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.294 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 22.292377144, 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.295 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:20:13.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:20:13.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:20:13.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.301 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.103702423, 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.302 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:20:13.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:20:13.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.08149908, 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:13.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:20:13.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:20:13.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.079802962, 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.312 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:20:13.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:20:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.060098174, 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.317 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:20:13.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:20:13.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.060074214, 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.323 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:13.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:20:13.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:20:13.328 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.019482669, 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:13.333 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.006708772, 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.337 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:24291 3.11 3:8dbf632a:::rbd_data.1182bea5217b.0000000000000111:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.791108+0000", 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 12.442244565999999, 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013805260999999999, 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.338 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 24291 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791108+0000", 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791108+0000", 2024-05-08T03:20:13.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791109+0000", 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.99e-07 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791113+0000", 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8840000000000004e-06 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791122+0000", 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6729999999999998e-06 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791128+0000", 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8470000000000001e-06 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791168+0000", 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0766999999999998e-05 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.791250+0000", 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1957000000000002e-05 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.793392+0000", 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002141609 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.804809+0000", 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011417158 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.804845+0000", 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6409999999999999e-05 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.804883+0000", 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7827000000000002e-05 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.804914+0000", 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0429999999999998e-05 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.344 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:13.346 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:13.347 INFO:teuthology.orchestra.run.smithi177.stdout:2705 trunc from 0xc9b2b37 to 0x7967667 2024-05-08T03:20:13.347 INFO:teuthology.orchestra.run.smithi177.stdout:2706 read 0x2d2aba0 thru 0x2d3a08a (0xf4eb bytes) 2024-05-08T03:20:13.347 INFO:teuthology.orchestra.run.smithi177.stdout:2707 writesame 0xcbe1b76 thru 0xcbe22c6 data_size 0x1(0x751 bytes) 2024-05-08T03:20:13.347 INFO:teuthology.orchestra.run.smithi177.stdout:2708 write 0x9f77603 thru 0x9f84091 (0xca8f bytes) 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:2709 read 0xa485c62 thru 0xa485dbc (0x15b bytes) 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:2710 writesame 0xbf7b1a3 thru 0xbf87824 data_size 0x1(0xc682 bytes) 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:2714 write 0x7ee9b0d thru 0x7ee9faa (0x49e bytes) 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:2715 write 0x9170bcf thru 0x917ba5d (0xae8f bytes) 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:2718 read 0x1e28a02 thru 0x1e30259 (0x7858 bytes) 2024-05-08T03:20:13.348 INFO:teuthology.orchestra.run.smithi177.stdout:2719 read 0x8ceac1d thru 0x8cf9558 (0xe93c bytes) 2024-05-08T03:20:13.349 INFO:teuthology.orchestra.run.smithi177.stdout:2720 read 0x9631477 thru 0x963428d (0x2e17 bytes) 2024-05-08T03:20:13.349 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.349 INFO:teuthology.orchestra.run.smithi177.stdout:2723 writesame 0x1e7e864 thru 0x1e82e8a data_size 0x1(0x4627 bytes) 2024-05-08T03:20:13.349 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:13.344+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:13.350 INFO:teuthology.orchestra.run.smithi177.stdout:2724 punch from 0x4f3be50 to 0x4f484b9, (0xc669 bytes) 2024-05-08T03:20:13.350 INFO:teuthology.orchestra.run.smithi177.stdout:2726 read 0x3b763c4 thru 0x3b7d480 (0x70bd bytes) 2024-05-08T03:20:13.355 INFO:teuthology.orchestra.run.smithi177.stdout:2727 write 0xebd2bc8 thru 0xebd9829 (0x6c62 bytes) 2024-05-08T03:20:13.364 INFO:teuthology.orchestra.run.smithi177.stdout:2728 write 0xe90785d thru 0xe90ceb7 (0x565b bytes) 2024-05-08T03:20:13.371 INFO:teuthology.orchestra.run.smithi177.stdout:2730 write 0xdadfbcc thru 0xdae74a1 (0x78d6 bytes) 2024-05-08T03:20:13.380 INFO:teuthology.orchestra.run.smithi177.stdout:2731 write 0x6b20868 thru 0x6b27fa4 (0x773d bytes) 2024-05-08T03:20:13.387 INFO:teuthology.orchestra.run.smithi177.stdout:2732 writesame 0x62364bc thru 0x6238343 data_size 0x4(0x1e88 bytes) 2024-05-08T03:20:13.391 INFO:teuthology.orchestra.run.smithi177.stdout:2733 write 0x1c045f7 thru 0x1c09106 (0x4b10 bytes) 2024-05-08T03:20:13.426 INFO:teuthology.orchestra.run.smithi177.stdout:2734 read 0x4f4f9fc thru 0x4f5e9b6 (0xefbb bytes) 2024-05-08T03:20:13.427 INFO:teuthology.orchestra.run.smithi177.stdout:2735 punch from 0x691a15 to 0x69848c, (0x6a77 bytes) 2024-05-08T03:20:13.427 INFO:teuthology.orchestra.run.smithi177.stdout:2736 write 0xc0b9868 thru 0xc0c2f4e (0x96e7 bytes) 2024-05-08T03:20:13.438 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.438 INFO:teuthology.orchestra.run.smithi177.stdout:2741 read 0x2f60d82 thru 0x2f6bbf6 (0xae75 bytes) 2024-05-08T03:20:13.439 INFO:teuthology.orchestra.run.smithi177.stdout:2743 read 0x5f8ff31 thru 0x5f92a4c (0x2b1c bytes) 2024-05-08T03:20:13.439 INFO:teuthology.orchestra.run.smithi177.stdout:2744 writesame 0x1487c9a thru 0x14972da data_size 0xb(0xf641 bytes) 2024-05-08T03:20:13.446 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:13.444+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:13.496 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:13.496 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:13.496 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:13.496 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:13.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:13.515 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.515 INFO:teuthology.orchestra.run.smithi177.stdout:2746 writesame 0xbad477d thru 0xbad70a4 data_size 0xc(0x2928 bytes) 2024-05-08T03:20:13.516 INFO:teuthology.orchestra.run.smithi177.stdout:2747 write 0x19b3043 thru 0x19c0837 (0xd7f5 bytes) 2024-05-08T03:20:13.536 INFO:teuthology.orchestra.run.smithi177.stdout:2749 write 0xb53f06 thru 0xb5e8d7 (0xa9d2 bytes) 2024-05-08T03:20:13.548 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:13.545+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:13.597 INFO:teuthology.orchestra.run.smithi177.stdout:2750 punch from 0x289087e to 0x289d31b, (0xca9d bytes) 2024-05-08T03:20:13.597 INFO:teuthology.orchestra.run.smithi177.stdout:2752 punch from 0x22be329 to 0x22c5565, (0x723c bytes) 2024-05-08T03:20:13.597 INFO:teuthology.orchestra.run.smithi177.stdout:2753 punch from 0xe344c3b to 0xe34aa3c, (0x5e01 bytes) 2024-05-08T03:20:13.601 INFO:teuthology.orchestra.run.smithi177.stdout:2754 trunc from 0xebd982a to 0xb06a760 2024-05-08T03:20:13.649 INFO:teuthology.orchestra.run.smithi177.stdout:2755 punch from 0x43d088f to 0x43e0586, (0xfcf7 bytes) 2024-05-08T03:20:13.649 INFO:teuthology.orchestra.run.smithi177.stdout:2756 trunc from 0xb06a760 to 0x41fe0c6 2024-05-08T03:20:13.649 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:13.649 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:13.649 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:13.650 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:13.650 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:13.650 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:13.650 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:13.647+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:13.720 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:13.720 INFO:teuthology.orchestra.run.smithi177.stdout:2759 clone 24 order 21 su 8192 sc 6 2024-05-08T03:20:13.750 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:13.748+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:13.797 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:13.797 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:13.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:13.797 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.920935667999998, 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.798 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:20:13.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:20:13.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:20:13.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:13.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:20:13.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578994475999998, 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.804 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.808 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578979077, 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:20:13.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578947435000003, 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.813 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:13.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:20:13.817 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578934103000002, 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.818 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578924309000001, 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.822 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.826 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578914896000001, 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.827 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:20:13.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578905374999998, 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:13.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578896034000003, 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:20:13.836 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:13.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:20:13.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:20:13.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:20:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578886818999997, 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:20:13.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578877456999997, 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:20:13.845 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:13.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:20:13.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:20:13.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578868024999998, 2024-05-08T03:20:13.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:13.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:20:13.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:20:13.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.854 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578858609000001, 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.855 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:20:13.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.859 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578849591000001, 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:13.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.863 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578840548000002, 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.864 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:20:13.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:13.868 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578828254999998, 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.869 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:20:13.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578818988000002, 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:20:13.873 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:20:13.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:20:13.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:20:13.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578805129999999, 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.878 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:20:13.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:20:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.578795837000001, 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:20:13.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:13.887 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.585781703999999, 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:13.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:20:13.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:20:13.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:13.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:13.892 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6] out_osds: [4, 7, 0, 5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:13.892 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:13.892 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T03:20:13.893 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:20:13.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:13.895 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:13.893+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:13.995 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:13.994+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:14.003 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:14.003 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:14.003 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:14.003 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:14.013 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:14.077 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:20:14.136 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:14.135+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:14.140 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone23 from 0x41fe0c6 (overlap 0x41fe0c6) to 0x20ff063 2024-05-08T03:20:14.199 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:14.199 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:14.199 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:14.199 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:14.199 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:14.200 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:14.198+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:14.213 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 66, 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:20:14.069156+0000", 2024-05-08T03:20:14.276 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:20:08.032979+0000", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:19:55.957014+0000", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5804032, 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "nodeep-scrub", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:20:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 5, 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:20:14.278 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "luminous", 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:14.279 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:20:14.280 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:14.281 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:14.282 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:14.283 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:14.284 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:14.285 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.630000114440918, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.630000114440918, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.56999999284744263, 2024-05-08T03:20:14.286 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.5, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.5, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:14.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:14.288 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:20:14.289 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:20:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:14.291 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:14.292 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:14.293 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 6.9000000953674316, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 6.9000000953674316, 2024-05-08T03:20:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:20:14.295 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:20:14.296 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "66", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 26, 2024-05-08T03:20:14.297 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 66, 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:14.298 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:14.299 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.8599998950958252, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.8599998950958252, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.56999999284744263, 2024-05-08T03:20:14.300 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.6299999952316284, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.6299999952316284, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:14.301 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 40, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.306 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 60, 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.309 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.310 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.310 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:20:14.310 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.311 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.311 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:20:14.311 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.311 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.311 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.313 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:14.314 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 61, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.316 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.318 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.319 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:14.320 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 56, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.322 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.323 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.324 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:20:14.325 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.327 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 19, 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.328 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.329 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.331 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 0, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 45, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 61, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.334 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.335 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:20:14.336 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1643429991", 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1643429991", 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1643429991", 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1643429991", 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: "exists" 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 54, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.338 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.339 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:20:14.340 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:20:14.341 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:20:14.342 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:14.343 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:14.345 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:20:14.346 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.347 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:14.349 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.350 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.351 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.353 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:20:14.354 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 61 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:14.355 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [ 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.d", 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:20:14.356 INFO:teuthology.orchestra.run.smithi177.stdout: 6 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [ 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.6", 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 1, 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:14.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.12", 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 3, 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.17", 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 3, 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:20:14.359 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [], 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [], 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:20:14.360 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.361 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:14.363 INFO:tasks.thrashosds.thrasher:j is {'epoch': 66, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:20:14.069156+0000', 'last_up_change': '2024-05-08T03:20:08.032979+0000', 'last_in_change': '2024-05-08T03:19:55.957014+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 0.5699999928474426, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.859999895095825, 'score_stable': 2.859999895095825, 'optimal_score': 0.5699999928474426, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 45, 'down_at': 61, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1643429991}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1643429991}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1643429991}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1643429991}]}, 'public_addr': '172.21.15.188:6817/1643429991', 'cluster_addr': '172.21.15.188:6819/1643429991', 'heartbeat_back_addr': '172.21.15.188:6823/1643429991', 'heartbeat_front_addr': '172.21.15.188:6821/1643429991', 'state': ['exists']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 61}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.d', 'osds': [3, 6]}], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '3.12', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.17', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:20:14.364 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:20:14.364 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:14.360+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:14.440 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:14.440 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.237875234999997, 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.441 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:14.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.237865784999997, 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.445 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:14.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.237853049000002, 2024-05-08T03:20:14.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.455 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.237842473000001, 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:14.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.237833086000002, 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.461 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.237823906000003, 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:14.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:14.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006998871, 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.470 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:14.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006966634000001, 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:14.475 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.476 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.476 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.476 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.476 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.476 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.476 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:14.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:14.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006953304, 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.482 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:14.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006938329999997, 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:14.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:14.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006927021000003, 2024-05-08T03:20:14.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:14.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:14.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:14.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.496 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006909352000001, 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.497 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:14.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:14.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.504 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.504 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:14.504 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.006894154000001, 2024-05-08T03:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:14.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:14.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:14.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:14.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:14.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:14.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:14.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:14.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:14.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:14.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:14.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:14.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:14.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:14.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.513 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.00688006, 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.514 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 38.231926086999998, 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:14.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:14.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:14.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.309032285000001, 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:14.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:14.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:14.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.309018797, 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:14.529 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:14.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:14.533 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.308989816, 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:14.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.308963210000002, 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:14.542 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.876214949000001, 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:14.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:14.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:14.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.548 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:14.550 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:14.550 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-05-08T03:20:14.550 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:14.548+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:14.624 INFO:teuthology.orchestra.run.smithi177.stdout:2760 write 0xa522700 thru 0xa528bdb (0x64dc bytes) 2024-05-08T03:20:14.632 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.632 INFO:teuthology.orchestra.run.smithi177.stdout:2763 read 0x50001f1 thru 0x5000fa8 (0xdb8 bytes) 2024-05-08T03:20:14.632 INFO:teuthology.orchestra.run.smithi177.stdout:2764 punch from 0x4dbfb06 to 0x4dcd35a, (0xd854 bytes) 2024-05-08T03:20:14.633 INFO:teuthology.orchestra.run.smithi177.stdout:2766 writesame 0x94aba8d thru 0x94bb6d9 data_size 0x1(0xfc4d bytes) 2024-05-08T03:20:14.650 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:14.648+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:14.657 INFO:teuthology.orchestra.run.smithi177.stdout:2767 read 0x3c9e723 thru 0x3ca4700 (0x5fde bytes) 2024-05-08T03:20:14.658 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.658 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.658 INFO:teuthology.orchestra.run.smithi177.stdout:2770 write 0x79572ef thru 0x795a5f1 (0x3303 bytes) 2024-05-08T03:20:14.663 INFO:teuthology.orchestra.run.smithi177.stdout:2772 write 0x3e0374a thru 0x3e05509 (0x1dc0 bytes) 2024-05-08T03:20:14.684 INFO:teuthology.orchestra.run.smithi177.stdout:2774 read 0x5880be0 thru 0x588a077 (0x9498 bytes) 2024-05-08T03:20:14.684 INFO:teuthology.orchestra.run.smithi177.stdout:2775 read 0x9d074b0 thru 0x9d13051 (0xbba2 bytes) 2024-05-08T03:20:14.684 INFO:teuthology.orchestra.run.smithi177.stdout:2776 punch from 0x32d1a2f to 0x32d47dd, (0x2dae bytes) 2024-05-08T03:20:14.685 INFO:teuthology.orchestra.run.smithi177.stdout:2777 punch from 0x732af57 to 0x7337212, (0xc2bb bytes) 2024-05-08T03:20:14.685 INFO:teuthology.orchestra.run.smithi177.stdout:2778 read 0x91202a7 thru 0x912a78d (0xa4e7 bytes) 2024-05-08T03:20:14.685 INFO:teuthology.orchestra.run.smithi177.stdout:2779 read 0x48ea008 thru 0x48edb87 (0x3b80 bytes) 2024-05-08T03:20:14.689 INFO:teuthology.orchestra.run.smithi177.stdout:2780 trunc from 0xa528bdc to 0xcf30626 2024-05-08T03:20:14.696 INFO:teuthology.orchestra.run.smithi177.stdout:2781 trunc from 0xcf30626 to 0xecc1f60 2024-05-08T03:20:14.702 INFO:teuthology.orchestra.run.smithi177.stdout:2782 read 0x3c505b3 thru 0x3c5d51c (0xcf6a bytes) 2024-05-08T03:20:14.703 INFO:teuthology.orchestra.run.smithi177.stdout:2783 read 0x43c6125 thru 0x43cb1c3 (0x509f bytes) 2024-05-08T03:20:14.704 INFO:teuthology.orchestra.run.smithi177.stdout:2784 read 0x8c2f006 thru 0x8c2fb60 (0xb5b bytes) 2024-05-08T03:20:14.704 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.704 INFO:teuthology.orchestra.run.smithi177.stdout:2787 punch from 0xb45fd10 to 0xb4642b6, (0x45a6 bytes) 2024-05-08T03:20:14.704 INFO:teuthology.orchestra.run.smithi177.stdout:2788 read 0x8ca6bf2 thru 0x8caed17 (0x8126 bytes) 2024-05-08T03:20:14.704 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.704 INFO:teuthology.orchestra.run.smithi177.stdout:2790 punch from 0xe494776 to 0xe496a6e, (0x22f8 bytes) 2024-05-08T03:20:14.705 INFO:teuthology.orchestra.run.smithi177.stdout:2791 write 0x6758400 thru 0x675df3a (0x5b3b bytes) 2024-05-08T03:20:14.716 INFO:teuthology.orchestra.run.smithi177.stdout:2794 trunc from 0xecc1f60 to 0x9af3232 2024-05-08T03:20:14.723 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:47680 3.a 3:522f1308:::rbd_data.1182fdc479eb.000000000000004a:head [truncate 1916928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:14.721877+0000", 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00067876500000000001, 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069696299999999999, 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for sub ops", 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:14.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 47680 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721877+0000", 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721877+0000", 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721879+0000", 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9999999999999999e-06 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721886+0000", 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8630000000000002e-06 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721904+0000", 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7861999999999999e-05 2024-05-08T03:20:14.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721912+0000", 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9680000000000006e-06 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.721992+0000", 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9513999999999998e-05 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.722094+0000", 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000102681 2024-05-08T03:20:14.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:14.722225+0000", 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013060799999999999 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:20:14.728 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:14.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:14.744 INFO:teuthology.orchestra.run.smithi177.stdout:2795 writesame 0xe59724d thru 0xe5a0771 data_size 0x59(0x9525 bytes) 2024-05-08T03:20:14.756 INFO:teuthology.orchestra.run.smithi177.stdout:2796 trunc from 0xe5a0772 to 0x40cf681 2024-05-08T03:20:14.757 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:14.755+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:14.761 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:14.761 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:20:14.783 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-05-08T03:20:14.183473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1042016251,"num_objects":1076,"num_object_clones":221,"num_object_copies":2152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1076,"num_whiteouts":386,"num_read":50463,"num_read_kb":451229,"num_write":7976,"num_write_kb":379185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1135,"num_bytes_recovered":161555472,"num_keys_recovered":229,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6491,"ondisk_log_size":6491,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":738308,"kb_used_data":584428,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":376749052,"statfs":{"total":386547056640,"available":385791029248,"internally_reserved":0,"allocated":598454272,"data_stored":903019009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":157476631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28331859,"num_objects":189,"num_object_clones":73,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":14,"num_read":11252,"num_read_kb":40149,"num_write":1434,"num_write_kb":16726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":9270833,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.125177"},"pg_stats":[{"pgid":"3.1f","version":"62'114","reported_seq":852,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066340+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:20:11.066340+0000","last_peered":"2024-05-08T03:20:11.066340+0000","last_clean":"2024-05-08T03:20:11.066340+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:20:11.066340+0000","last_undegraded":"2024-05-08T03:20:11.066340+0000","last_fullsized":"2024-05-08T03:20:11.066340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:48.227767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":34679537,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":1021,"num_read_kb":6104,"num_write":141,"num_write_kb":4946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"63'114","reported_seq":478,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074736+0000","last_change":"2024-05-08T03:19:22.850001+0000","last_active":"2024-05-08T03:20:11.074736+0000","last_peered":"2024-05-08T03:20:11.074736+0000","last_clean":"2024-05-08T03:20:11.074736+0000","last_became_active":"2024-05-08T03:19:20.773128+0000","last_became_peered":"2024-05-08T03:19:20.773128+0000","last_unstale":"2024-05-08T03:20:11.074736+0000","last_undegraded":"2024-05-08T03:20:11.074736+0000","last_fullsized":"2024-05-08T03:20:11.074736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:17.542964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":21452273,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":344,"num_read_kb":507,"num_write":111,"num_write_kb":1007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"63'239","reported_seq":1412,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.102564+0000","last_change":"2024-05-08T03:20:03.022224+0000","last_active":"2024-05-08T03:20:11.102564+0000","last_peered":"2024-05-08T03:20:11.102564+0000","last_clean":"2024-05-08T03:20:11.102564+0000","last_became_active":"2024-05-08T03:20:03.021608+0000","last_became_peered":"2024-05-08T03:20:03.021608+0000","last_unstale":"2024-05-08T03:20:11.102564+0000","last_undegraded":"2024-05-08T03:20:11.102564+0000","last_fullsized":"2024-05-08T03:20:11.102564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:59.426390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":38040208,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":1899,"num_read_kb":2728,"num_write":302,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":278351,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"64'208","reported_seq":1875,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.196790+0000","last_change":"2024-05-08T03:19:59.997205+0000","last_active":"2024-05-08T03:20:11.196790+0000","last_peered":"2024-05-08T03:20:11.196790+0000","last_clean":"2024-05-08T03:20:11.196790+0000","last_became_active":"2024-05-08T03:19:59.996571+0000","last_became_peered":"2024-05-08T03:19:59.996571+0000","last_unstale":"2024-05-08T03:20:11.196790+0000","last_undegraded":"2024-05-08T03:20:11.196790+0000","last_fullsized":"2024-05-08T03:20:11.196790+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:07.895802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":40332787,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":2065,"num_read_kb":41624,"num_write":276,"num_write_kb":36478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":1994536,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"64'265","reported_seq":1606,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.177341+0000","last_change":"2024-05-08T03:19:51.215870+0000","last_active":"2024-05-08T03:20:11.177341+0000","last_peered":"2024-05-08T03:20:11.177341+0000","last_clean":"2024-05-08T03:20:11.177341+0000","last_became_active":"2024-05-08T03:19:50.935675+0000","last_became_peered":"2024-05-08T03:19:50.935675+0000","last_unstale":"2024-05-08T03:20:11.177341+0000","last_undegraded":"2024-05-08T03:20:11.177341+0000","last_fullsized":"2024-05-08T03:20:11.177341+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:21.344743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":26883872,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":2259,"num_read_kb":4307,"num_write":355,"num_write_kb":8426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2376803,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"63'345","reported_seq":2305,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.200104+0000","last_change":"2024-05-08T03:19:31.854145+0000","last_active":"2024-05-08T03:20:11.200104+0000","last_peered":"2024-05-08T03:20:11.200104+0000","last_clean":"2024-05-08T03:20:11.200104+0000","last_became_active":"2024-05-08T03:19:31.830295+0000","last_became_peered":"2024-05-08T03:19:31.830295+0000","last_unstale":"2024-05-08T03:20:11.200104+0000","last_undegraded":"2024-05-08T03:20:11.200104+0000","last_fullsized":"2024-05-08T03:20:11.200104+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:33:32.222267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":40440960,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":2840,"num_read_kb":32732,"num_write":506,"num_write_kb":34906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":91773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"63'123","reported_seq":949,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.067822+0000","last_change":"2024-05-08T03:19:22.779129+0000","last_active":"2024-05-08T03:20:11.067822+0000","last_peered":"2024-05-08T03:20:11.067822+0000","last_clean":"2024-05-08T03:20:11.067822+0000","last_became_active":"2024-05-08T03:19:20.775300+0000","last_became_peered":"2024-05-08T03:19:20.775300+0000","last_unstale":"2024-05-08T03:20:11.067822+0000","last_undegraded":"2024-05-08T03:20:11.067822+0000","last_fullsized":"2024-05-08T03:20:11.067822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:30.756962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":35706831,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":758,"num_read_kb":1680,"num_write":104,"num_write_kb":1142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"63'214","reported_seq":2384,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.211074+0000","last_change":"2024-05-08T03:19:53.957566+0000","last_active":"2024-05-08T03:20:11.211074+0000","last_peered":"2024-05-08T03:20:11.211074+0000","last_clean":"2024-05-08T03:20:11.211074+0000","last_became_active":"2024-05-08T03:19:53.957072+0000","last_became_peered":"2024-05-08T03:19:53.957072+0000","last_unstale":"2024-05-08T03:20:11.211074+0000","last_undegraded":"2024-05-08T03:20:11.211074+0000","last_fullsized":"2024-05-08T03:20:11.211074+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:46:33.523019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":45360152,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":3192,"num_read_kb":16161,"num_write":218,"num_write_kb":7226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":6126489,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"64'90","reported_seq":211,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.157045+0000","last_change":"2024-05-08T03:20:06.164893+0000","last_active":"2024-05-08T03:20:11.157045+0000","last_peered":"2024-05-08T03:20:11.157045+0000","last_clean":"2024-05-08T03:20:11.157045+0000","last_became_active":"2024-05-08T03:20:06.035685+0000","last_became_peered":"2024-05-08T03:20:06.035685+0000","last_unstale":"2024-05-08T03:20:11.157045+0000","last_undegraded":"2024-05-08T03:20:11.157045+0000","last_fullsized":"2024-05-08T03:20:11.157045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:12.365098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":8300767,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":145,"num_read_kb":667,"num_write":90,"num_write_kb":663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":192270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"63'95","reported_seq":1489,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.151701+0000","last_change":"2024-05-08T03:19:57.081242+0000","last_active":"2024-05-08T03:20:11.151701+0000","last_peered":"2024-05-08T03:20:11.151701+0000","last_clean":"2024-05-08T03:20:11.151701+0000","last_became_active":"2024-05-08T03:19:56.989658+0000","last_became_peered":"2024-05-08T03:19:56.989658+0000","last_unstale":"2024-05-08T03:20:11.151701+0000","last_undegraded":"2024-05-08T03:20:11.151701+0000","last_fullsized":"2024-05-08T03:20:11.151701+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:19:08.782058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":74362580,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":1231,"num_read_kb":41866,"num_write":89,"num_write_kb":38062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":8790475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"63'113","reported_seq":840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074715+0000","last_change":"2024-05-08T03:19:50.996510+0000","last_active":"2024-05-08T03:20:11.074715+0000","last_peered":"2024-05-08T03:20:11.074715+0000","last_clean":"2024-05-08T03:20:11.074715+0000","last_became_active":"2024-05-08T03:19:50.927921+0000","last_became_peered":"2024-05-08T03:19:50.927921+0000","last_unstale":"2024-05-08T03:20:11.074715+0000","last_undegraded":"2024-05-08T03:20:11.074715+0000","last_fullsized":"2024-05-08T03:20:11.074715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:12.032022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":25041592,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":851,"num_read_kb":1242,"num_write":93,"num_write_kb":441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":268258,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"63'390","reported_seq":1324,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.164199+0000","last_change":"2024-05-08T03:20:03.022563+0000","last_active":"2024-05-08T03:20:11.164199+0000","last_peered":"2024-05-08T03:20:11.164199+0000","last_clean":"2024-05-08T03:20:11.164199+0000","last_became_active":"2024-05-08T03:20:03.021688+0000","last_became_peered":"2024-05-08T03:20:03.021688+0000","last_unstale":"2024-05-08T03:20:11.164199+0000","last_undegraded":"2024-05-08T03:20:11.164199+0000","last_fullsized":"2024-05-08T03:20:11.164199+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:35:17.586545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":11720236,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":2470,"num_read_kb":9154,"num_write":552,"num_write_kb":3976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":135734,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074803+0000","last_change":"2024-05-08T03:20:03.022786+0000","last_active":"2024-05-08T03:20:11.074803+0000","last_peered":"2024-05-08T03:20:11.074803+0000","last_clean":"2024-05-08T03:20:11.074803+0000","last_became_active":"2024-05-08T03:20:03.022530+0000","last_became_peered":"2024-05-08T03:20:03.022530+0000","last_unstale":"2024-05-08T03:20:11.074803+0000","last_undegraded":"2024-05-08T03:20:11.074803+0000","last_fullsized":"2024-05-08T03:20:11.074803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:59.431605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"63'134","reported_seq":610,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074592+0000","last_change":"2024-05-08T03:19:57.064707+0000","last_active":"2024-05-08T03:20:11.074592+0000","last_peered":"2024-05-08T03:20:11.074592+0000","last_clean":"2024-05-08T03:20:11.074592+0000","last_became_active":"2024-05-08T03:19:56.989832+0000","last_became_peered":"2024-05-08T03:19:56.989832+0000","last_unstale":"2024-05-08T03:20:11.074592+0000","last_undegraded":"2024-05-08T03:20:11.074592+0000","last_fullsized":"2024-05-08T03:20:11.074592+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:28.303803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":6160692,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":345,"num_read_kb":3790,"num_write":127,"num_write_kb":1585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":208697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074757+0000","last_change":"2024-05-08T03:20:03.023919+0000","last_active":"2024-05-08T03:20:11.074757+0000","last_peered":"2024-05-08T03:20:11.074757+0000","last_clean":"2024-05-08T03:20:11.074757+0000","last_became_active":"2024-05-08T03:20:03.023611+0000","last_became_peered":"2024-05-08T03:20:03.023611+0000","last_unstale":"2024-05-08T03:20:11.074757+0000","last_undegraded":"2024-05-08T03:20:11.074757+0000","last_fullsized":"2024-05-08T03:20:11.074757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:04:35.966897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"64'400","reported_seq":1747,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.194494+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:20:11.194494+0000","last_peered":"2024-05-08T03:20:11.194494+0000","last_clean":"2024-05-08T03:20:11.194494+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:20:11.194494+0000","last_undegraded":"2024-05-08T03:20:11.194494+0000","last_fullsized":"2024-05-08T03:20:11.194494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:51.780146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":22854920,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":11,"num_read":3191,"num_read_kb":26359,"num_write":537,"num_write_kb":15906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066040+0000","last_change":"2024-05-08T03:19:31.832510+0000","last_active":"2024-05-08T03:20:11.066040+0000","last_peered":"2024-05-08T03:20:11.066040+0000","last_clean":"2024-05-08T03:20:11.066040+0000","last_became_active":"2024-05-08T03:19:31.832151+0000","last_became_peered":"2024-05-08T03:19:31.832151+0000","last_unstale":"2024-05-08T03:20:11.066040+0000","last_undegraded":"2024-05-08T03:20:11.066040+0000","last_fullsized":"2024-05-08T03:20:11.066040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:31.808173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"63'111","reported_seq":1606,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.252853+0000","last_change":"2024-05-08T03:19:57.108001+0000","last_active":"2024-05-08T03:20:11.252853+0000","last_peered":"2024-05-08T03:20:11.252853+0000","last_clean":"2024-05-08T03:20:11.252853+0000","last_became_active":"2024-05-08T03:19:56.991735+0000","last_became_peered":"2024-05-08T03:19:56.991735+0000","last_unstale":"2024-05-08T03:20:11.252853+0000","last_undegraded":"2024-05-08T03:20:11.252853+0000","last_fullsized":"2024-05-08T03:20:11.252853+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:43.838786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":4002158,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":1392,"num_read_kb":5268,"num_write":115,"num_write_kb":1101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":78741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074406+0000","last_change":"2024-05-08T03:19:56.989000+0000","last_active":"2024-05-08T03:20:11.074406+0000","last_peered":"2024-05-08T03:20:11.074406+0000","last_clean":"2024-05-08T03:20:11.074406+0000","last_became_active":"2024-05-08T03:19:56.988667+0000","last_became_peered":"2024-05-08T03:19:56.988667+0000","last_unstale":"2024-05-08T03:20:11.074406+0000","last_undegraded":"2024-05-08T03:20:11.074406+0000","last_fullsized":"2024-05-08T03:20:11.074406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:33.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'404","reported_seq":2266,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066731+0000","last_change":"2024-05-08T03:20:09.057346+0000","last_active":"2024-05-08T03:20:11.066731+0000","last_peered":"2024-05-08T03:20:11.066731+0000","last_clean":"2024-05-08T03:20:11.066731+0000","last_became_active":"2024-05-08T03:20:09.056416+0000","last_became_peered":"2024-05-08T03:20:09.056416+0000","last_unstale":"2024-05-08T03:20:11.066731+0000","last_undegraded":"2024-05-08T03:20:11.066731+0000","last_fullsized":"2024-05-08T03:20:11.066731+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:54:36.984995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":25464273,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":3054,"num_read_kb":23263,"num_write":634,"num_write_kb":25310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":30566047,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"63'92","reported_seq":635,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.074321+0000","last_change":"2024-05-08T03:19:57.175539+0000","last_active":"2024-05-08T03:20:11.074321+0000","last_peered":"2024-05-08T03:20:11.074321+0000","last_clean":"2024-05-08T03:20:11.074321+0000","last_became_active":"2024-05-08T03:19:56.994286+0000","last_became_peered":"2024-05-08T03:19:56.994286+0000","last_unstale":"2024-05-08T03:20:11.074321+0000","last_undegraded":"2024-05-08T03:20:11.074321+0000","last_fullsized":"2024-05-08T03:20:11.074321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:08.378997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":7304940,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":344,"num_read_kb":3761,"num_write":82,"num_write_kb":2754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1928876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":238,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.067550+0000","last_change":"2024-05-08T03:19:53.957635+0000","last_active":"2024-05-08T03:20:11.067550+0000","last_peered":"2024-05-08T03:20:11.067550+0000","last_clean":"2024-05-08T03:20:11.067550+0000","last_became_active":"2024-05-08T03:19:53.957252+0000","last_became_peered":"2024-05-08T03:19:53.957252+0000","last_unstale":"2024-05-08T03:20:11.067550+0000","last_undegraded":"2024-05-08T03:20:11.067550+0000","last_fullsized":"2024-05-08T03:20:11.067550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:27.307894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.068521+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:20:11.068521+0000","last_peered":"2024-05-08T03:20:11.068521+0000","last_clean":"2024-05-08T03:20:11.068521+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:20:11.068521+0000","last_undegraded":"2024-05-08T03:20:11.068521+0000","last_fullsized":"2024-05-08T03:20:11.068521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"63'126","reported_seq":884,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.138999+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:20:11.138999+0000","last_peered":"2024-05-08T03:20:11.138999+0000","last_clean":"2024-05-08T03:20:11.138999+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:20:11.138999+0000","last_undegraded":"2024-05-08T03:20:11.138999+0000","last_fullsized":"2024-05-08T03:20:11.138999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:12:57.809981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":15238619,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":1207,"num_read_kb":2867,"num_write":110,"num_write_kb":909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":54,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066513+0000","last_change":"2024-05-08T03:19:56.983410+0000","last_active":"2024-05-08T03:20:11.066513+0000","last_peered":"2024-05-08T03:20:11.066513+0000","last_clean":"2024-05-08T03:20:11.066513+0000","last_became_active":"2024-05-08T03:19:56.983063+0000","last_became_peered":"2024-05-08T03:19:56.983063+0000","last_unstale":"2024-05-08T03:20:11.066513+0000","last_undegraded":"2024-05-08T03:20:11.066513+0000","last_fullsized":"2024-05-08T03:20:11.066513+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:54:24.071706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"63'321","reported_seq":1226,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.068124+0000","last_change":"2024-05-08T03:20:03.022688+0000","last_active":"2024-05-08T03:20:11.068124+0000","last_peered":"2024-05-08T03:20:11.068124+0000","last_clean":"2024-05-08T03:20:11.068124+0000","last_became_active":"2024-05-08T03:20:03.022099+0000","last_became_peered":"2024-05-08T03:20:03.022099+0000","last_unstale":"2024-05-08T03:20:11.068124+0000","last_undegraded":"2024-05-08T03:20:11.068124+0000","last_fullsized":"2024-05-08T03:20:11.068124+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:00.057953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":24223301,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":2279,"num_read_kb":3707,"num_write":436,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":481376,"num_keys_recovered":30,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066438+0000","last_change":"2024-05-08T03:19:56.988049+0000","last_active":"2024-05-08T03:20:11.066438+0000","last_peered":"2024-05-08T03:20:11.066438+0000","last_clean":"2024-05-08T03:20:11.066438+0000","last_became_active":"2024-05-08T03:19:56.987737+0000","last_became_peered":"2024-05-08T03:19:56.987737+0000","last_unstale":"2024-05-08T03:20:11.066438+0000","last_undegraded":"2024-05-08T03:20:11.066438+0000","last_fullsized":"2024-05-08T03:20:11.066438+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:53.545130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"64'441","reported_seq":1585,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.242433+0000","last_change":"2024-05-08T03:19:59.997039+0000","last_active":"2024-05-08T03:20:11.242433+0000","last_peered":"2024-05-08T03:20:11.242433+0000","last_clean":"2024-05-08T03:20:11.242433+0000","last_became_active":"2024-05-08T03:19:59.996378+0000","last_became_peered":"2024-05-08T03:19:59.996378+0000","last_unstale":"2024-05-08T03:20:11.242433+0000","last_undegraded":"2024-05-08T03:20:11.242433+0000","last_fullsized":"2024-05-08T03:20:11.242433+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:42.096418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":21807865,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":2847,"num_read_kb":16716,"num_write":664,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":6220729,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.067253+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:20:11.067253+0000","last_peered":"2024-05-08T03:20:11.067253+0000","last_clean":"2024-05-08T03:20:11.067253+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:20:11.067253+0000","last_undegraded":"2024-05-08T03:20:11.067253+0000","last_fullsized":"2024-05-08T03:20:11.067253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"64'176","reported_seq":1157,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.272430+0000","last_change":"2024-05-08T03:19:22.850234+0000","last_active":"2024-05-08T03:20:11.272430+0000","last_peered":"2024-05-08T03:20:11.272430+0000","last_clean":"2024-05-08T03:20:11.272430+0000","last_became_active":"2024-05-08T03:19:20.773661+0000","last_became_peered":"2024-05-08T03:19:20.773661+0000","last_unstale":"2024-05-08T03:20:11.272430+0000","last_undegraded":"2024-05-08T03:20:11.272430+0000","last_fullsized":"2024-05-08T03:20:11.272430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:19:48.181208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":6434848,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":2218,"num_read_kb":5402,"num_write":167,"num_write_kb":2014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"63'158","reported_seq":1106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066572+0000","last_change":"2024-05-08T03:19:44.872224+0000","last_active":"2024-05-08T03:20:11.066572+0000","last_peered":"2024-05-08T03:20:11.066572+0000","last_clean":"2024-05-08T03:20:11.066572+0000","last_became_active":"2024-05-08T03:19:44.774602+0000","last_became_peered":"2024-05-08T03:19:44.774602+0000","last_unstale":"2024-05-08T03:20:11.066572+0000","last_undegraded":"2024-05-08T03:20:11.066572+0000","last_fullsized":"2024-05-08T03:20:11.066572+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:35.586696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":76521941,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":820,"num_read_kb":10982,"num_write":190,"num_write_kb":14070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":5551754,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"64'164","reported_seq":573,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.068135+0000","last_change":"2024-05-08T03:20:03.022267+0000","last_active":"2024-05-08T03:20:11.068135+0000","last_peered":"2024-05-08T03:20:11.068135+0000","last_clean":"2024-05-08T03:20:11.068135+0000","last_became_active":"2024-05-08T03:20:03.021701+0000","last_became_peered":"2024-05-08T03:20:03.021701+0000","last_unstale":"2024-05-08T03:20:11.068135+0000","last_undegraded":"2024-05-08T03:20:11.068135+0000","last_fullsized":"2024-05-08T03:20:11.068135+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:28.044374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":23773269,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":579,"num_read_kb":7296,"num_write":195,"num_write_kb":4213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":24576,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"63'132","reported_seq":988,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.102926+0000","last_change":"2024-05-08T03:19:51.297822+0000","last_active":"2024-05-08T03:20:11.102926+0000","last_peered":"2024-05-08T03:20:11.102926+0000","last_clean":"2024-05-08T03:20:11.102926+0000","last_became_active":"2024-05-08T03:19:50.937736+0000","last_became_peered":"2024-05-08T03:19:50.937736+0000","last_unstale":"2024-05-08T03:20:11.102926+0000","last_undegraded":"2024-05-08T03:20:11.102926+0000","last_fullsized":"2024-05-08T03:20:11.102926+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:36.927886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":39232002,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":883,"num_read_kb":5845,"num_write":121,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2233399,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"63'257","reported_seq":1381,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.272067+0000","last_change":"2024-05-08T03:19:59.995470+0000","last_active":"2024-05-08T03:20:11.272067+0000","last_peered":"2024-05-08T03:20:11.272067+0000","last_clean":"2024-05-08T03:20:11.272067+0000","last_became_active":"2024-05-08T03:19:59.994538+0000","last_became_peered":"2024-05-08T03:19:59.994538+0000","last_unstale":"2024-05-08T03:20:11.272067+0000","last_undegraded":"2024-05-08T03:20:11.272067+0000","last_fullsized":"2024-05-08T03:20:11.272067+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:33:22.872719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":78328304,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":2290,"num_read_kb":12056,"num_write":333,"num_write_kb":14197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":8235378,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"64'238","reported_seq":736,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.272047+0000","last_change":"2024-05-08T03:19:39.181175+0000","last_active":"2024-05-08T03:20:11.272047+0000","last_peered":"2024-05-08T03:20:11.272047+0000","last_clean":"2024-05-08T03:20:11.272047+0000","last_became_active":"2024-05-08T03:19:39.090307+0000","last_became_peered":"2024-05-08T03:19:39.090307+0000","last_unstale":"2024-05-08T03:20:11.272047+0000","last_undegraded":"2024-05-08T03:20:11.272047+0000","last_fullsized":"2024-05-08T03:20:11.272047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:11.421319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":56044030,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":2155,"num_read_kb":34455,"num_write":295,"num_write_kb":34845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":993485,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"64'198","reported_seq":2150,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.169263+0000","last_change":"2024-05-08T03:19:59.997892+0000","last_active":"2024-05-08T03:20:11.169263+0000","last_peered":"2024-05-08T03:20:11.169263+0000","last_clean":"2024-05-08T03:20:11.169263+0000","last_became_active":"2024-05-08T03:19:59.997367+0000","last_became_peered":"2024-05-08T03:19:59.997367+0000","last_unstale":"2024-05-08T03:20:11.169263+0000","last_undegraded":"2024-05-08T03:20:11.169263+0000","last_fullsized":"2024-05-08T03:20:11.169263+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:09:01.083431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":53510216,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":1848,"num_read_kb":25968,"num_write":212,"num_write_kb":11875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":31431070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"64'138","reported_seq":951,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.162571+0000","last_change":"2024-05-08T03:19:57.244251+0000","last_active":"2024-05-08T03:20:11.162571+0000","last_peered":"2024-05-08T03:20:11.162571+0000","last_clean":"2024-05-08T03:20:11.162571+0000","last_became_active":"2024-05-08T03:19:56.989657+0000","last_became_peered":"2024-05-08T03:19:56.989657+0000","last_unstale":"2024-05-08T03:20:11.162571+0000","last_undegraded":"2024-05-08T03:20:11.162571+0000","last_fullsized":"2024-05-08T03:20:11.162571+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:01.946675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":12207535,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":571,"num_read_kb":3836,"num_write":146,"num_write_kb":7909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4608126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"64'105","reported_seq":801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.186295+0000","last_change":"2024-05-08T03:20:03.024129+0000","last_active":"2024-05-08T03:20:11.186295+0000","last_peered":"2024-05-08T03:20:11.186295+0000","last_clean":"2024-05-08T03:20:11.186295+0000","last_became_active":"2024-05-08T03:20:03.023616+0000","last_became_peered":"2024-05-08T03:20:03.023616+0000","last_unstale":"2024-05-08T03:20:11.186295+0000","last_undegraded":"2024-05-08T03:20:11.186295+0000","last_fullsized":"2024-05-08T03:20:11.186295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:58.630876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":56432384,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":526,"num_read_kb":17612,"num_write":121,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":10514324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"64'160","reported_seq":1782,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.273301+0000","last_change":"2024-05-08T03:20:03.024587+0000","last_active":"2024-05-08T03:20:11.273301+0000","last_peered":"2024-05-08T03:20:11.273301+0000","last_clean":"2024-05-08T03:20:11.273301+0000","last_became_active":"2024-05-08T03:20:03.023621+0000","last_became_peered":"2024-05-08T03:20:03.023621+0000","last_unstale":"2024-05-08T03:20:11.273301+0000","last_undegraded":"2024-05-08T03:20:11.273301+0000","last_fullsized":"2024-05-08T03:20:11.273301+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:27.343775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":52650958,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":1949,"num_read_kb":43464,"num_write":141,"num_write_kb":47400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":14483473,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"64'114","reported_seq":252,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.137898+0000","last_change":"2024-05-08T03:20:06.097542+0000","last_active":"2024-05-08T03:20:11.137898+0000","last_peered":"2024-05-08T03:20:11.137898+0000","last_clean":"2024-05-08T03:20:11.137898+0000","last_became_active":"2024-05-08T03:20:06.033352+0000","last_became_peered":"2024-05-08T03:20:06.033352+0000","last_unstale":"2024-05-08T03:20:11.137898+0000","last_undegraded":"2024-05-08T03:20:11.137898+0000","last_fullsized":"2024-05-08T03:20:11.137898+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:43.142833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":35426366,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":1564,"num_read_kb":12594,"num_write":102,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"64'195","reported_seq":1167,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T03:20:11.066580+0000","last_change":"2024-05-08T03:19:47.769634+0000","last_active":"2024-05-08T03:20:11.066580+0000","last_peered":"2024-05-08T03:20:11.066580+0000","last_clean":"2024-05-08T03:20:11.066580+0000","last_became_active":"2024-05-08T03:19:47.769070+0000","last_became_peered":"2024-05-08T03:19:47.769070+0000","last_unstale":"2024-05-08T03:20:11.066580+0000","last_undegraded":"2024-05-08T03:20:11.066580+0000","last_fullsized":"2024-05-08T03:20:11.066580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:34.131594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":21485467,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":1210,"num_read_kb":27116,"num_write":252,"num_write_kb":18396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":13456776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1041425883,"num_objects":1074,"num_object_clones":221,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1074,"num_whiteouts":386,"num_read":50347,"num_read_kb":451129,"num_write":7817,"num_write_kb":376769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1131,"num_bytes_recovered":160374736,"num_keys_recovered":229,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705970176,"data_stored":1058386565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6374,"ondisk_log_size":6374,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542159,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":210672,"kb_used_data":171164,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94161168,"statfs":{"total":96636764160,"available":96421036032,"internally_reserved":0,"allocated":175271936,"data_stored":270458346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":60129542159,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542160,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206284,"kb_used_data":166784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94165556,"statfs":{"total":96636764160,"available":96425529344,"internally_reserved":0,"allocated":170786816,"data_stored":252818122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122308,"kb_used_data":86964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94249532,"statfs":{"total":96636764160,"available":96511520768,"internally_reserved":0,"allocated":89051136,"data_stored":138638524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":199044,"kb_used_data":159516,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94172796,"statfs":{"total":96636764160,"available":96432943104,"internally_reserved":0,"allocated":163344384,"data_stored":241104017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542160,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165564416,"data_stored":243722559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":88326144,"data_stored":138308518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":171352064,"data_stored":253791997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":496740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":174551040,"data_stored":270128356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:20:14.785 INFO:tasks.thrashosds.thrasher:Setting 1.5 to [2, 0] 2024-05-08T03:20:14.785 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.5', '2', '0'] 2024-05-08T03:20:14.785 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.5 2 0 2024-05-08T03:20:14.808 INFO:teuthology.orchestra.run.smithi177.stdout:2797 punch from 0x35194b9 to 0x3519a00, (0x547 bytes) 2024-05-08T03:20:14.819 INFO:teuthology.orchestra.run.smithi177.stdout:2798 write 0x929e9f8 thru 0x92a3258 (0x4861 bytes) 2024-05-08T03:20:14.827 INFO:teuthology.orchestra.run.smithi177.stdout:2799 read 0x325a036 thru 0x32638c1 (0x988c bytes) 2024-05-08T03:20:14.828 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.828 INFO:teuthology.orchestra.run.smithi177.stdout:2801 read 0x242a4f4 thru 0x242fa89 (0x5596 bytes) 2024-05-08T03:20:14.828 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.831 INFO:teuthology.orchestra.run.smithi177.stdout:2803 trunc from 0x92a3259 to 0x6ed7f64 2024-05-08T03:20:14.844 INFO:teuthology.orchestra.run.smithi177.stdout:2804 punch from 0x3c473a8 to 0x3c49de2, (0x2a3a bytes) 2024-05-08T03:20:14.844 INFO:teuthology.orchestra.run.smithi177.stdout:2805 read 0x477c050 thru 0x478b7b6 (0xf767 bytes) 2024-05-08T03:20:14.844 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.844 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:14.850 INFO:teuthology.orchestra.run.smithi177.stdout:2810 trunc from 0x6ed7f64 to 0x2683304 2024-05-08T03:20:14.852 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:14.851+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:14.900 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:14.900 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:14.901 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:14.901 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:14.901 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:14.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:14.928 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:14.953 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:14.951+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:15.002 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.015 INFO:teuthology.orchestra.run.smithi177.stdout:2813 write 0x7d8ab52 thru 0x7d8f6ee (0x4b9d bytes) 2024-05-08T03:20:15.024 INFO:teuthology.orchestra.run.smithi177.stdout:2814 punch from 0x36ca618 to 0x36d113f, (0x6b27 bytes) 2024-05-08T03:20:15.031 INFO:teuthology.orchestra.run.smithi177.stdout:2815 trunc from 0x7d8f6ef to 0xdb6d763 2024-05-08T03:20:15.036 INFO:teuthology.orchestra.run.smithi177.stdout:2817 write 0xa9d38f2 thru 0xa9e3741 (0xfe50 bytes) 2024-05-08T03:20:15.048 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.048 INFO:teuthology.orchestra.run.smithi177.stdout:2820 write 0x5a0a534 thru 0x5a152dd (0xadaa bytes) 2024-05-08T03:20:15.055 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:15.053+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:15.056 INFO:teuthology.orchestra.run.smithi177.stdout:2821 writesame 0xd10fb6c thru 0xd115104 data_size 0x1(0x5599 bytes) 2024-05-08T03:20:15.068 INFO:teuthology.orchestra.run.smithi177.stdout:2822 read 0xafa3f29 thru 0xafac951 (0x8a29 bytes) 2024-05-08T03:20:15.068 INFO:teuthology.orchestra.run.smithi177.stdout:2823 read 0xb6971c7 thru 0xb6a4a25 (0xd85f bytes) 2024-05-08T03:20:15.077 INFO:teuthology.orchestra.run.smithi177.stdout:2824 trunc from 0xdb6d763 to 0x6f07c65 2024-05-08T03:20:15.097 INFO:teuthology.orchestra.run.smithi177.stdout:2825 punch from 0x6e9b05 to 0x6f07a8, (0x6ca3 bytes) 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:15.103 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 29.314806135000001, 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 25.167571985999999, 2024-05-08T03:20:15.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.977431819, 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.116 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:15.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:15.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:15.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:15.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:15.121 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.952691272999999, 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:15.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:15.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:15.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:15.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25865 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [sparse-read 0~19262493] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 11.630824072999999, 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053445962, 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.127 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25865 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469279+0000", 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.852e-06 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469291+0000", 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2098e-05 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469313+0000", 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2657999999999999e-05 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469326+0000", 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2782000000000001e-05 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469401+0000", 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4759999999999996e-05 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469495+0000", 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4000999999999997e-05 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.522722+0000", 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053226810999999999 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25866 3.1a 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 11.630202833, 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049237975000000003, 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.131 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25866 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:15.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469898+0000", 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.23e-07 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469902+0000", 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4839999999999998e-06 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469913+0000", 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0652999999999999e-05 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469926+0000", 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3010000000000001e-05 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470021+0000", 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5622e-05 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470105+0000", 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3938999999999997e-05 2024-05-08T03:20:15.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.519135+0000", 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049029544000000001 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25871 3.16 3:692ca2ad:::rbd_data.118227bb2f9.0000000000000006:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 11.630016063999999, 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.047103610999999997, 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.135 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25871 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.06e-07 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470092+0000", 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8310000000000001e-06 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470099+0000", 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0999999999999998e-06 2024-05-08T03:20:15.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470108+0000", 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3190000000000001e-06 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470162+0000", 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3618000000000001e-05 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470234+0000", 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2342999999999995e-05 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.517187+0000", 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046952894000000002 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27760 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 557056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.920879064999999, 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046225914999999999, 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.139 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27760 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8900000000000001e-07 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179222+0000", 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.114e-06 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179225+0000", 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9189999999999999e-06 2024-05-08T03:20:15.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179228+0000", 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.819e-06 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225336+0000", 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046108300999999997 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225388+0000", 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2057e-05 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225447+0000", 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8415999999999998e-05 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27761 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.920869165999999, 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046341465999999998, 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.143 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27761 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:15.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179232+0000", 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179235+0000", 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8770000000000002e-06 2024-05-08T03:20:15.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179237+0000", 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3690000000000001e-06 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225475+0000", 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046237568 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225515+0000", 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0080000000000003e-05 2024-05-08T03:20:15.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225572+0000", 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7351e-05 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27762 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 581632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.920860095, 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046463320000000002, 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.147 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27762 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.29e-07 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179241+0000", 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.263e-06 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179244+0000", 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7769999999999998e-06 2024-05-08T03:20:15.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179246+0000", 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5509999999999998e-06 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225609+0000", 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046362154000000003 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225650+0000", 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:15.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225703+0000", 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3260999999999997e-05 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27763 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.920850517, 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046573208999999997, 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.151 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27763 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:15.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179251+0000", 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.102e-06 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179253+0000", 2024-05-08T03:20:15.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8229999999999999e-06 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179256+0000", 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4949999999999998e-06 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225727+0000", 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046471512999999999 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225768+0000", 2024-05-08T03:20:15.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0142999999999998e-05 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225822+0000", 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4926000000000002e-05 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27764 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.920841179, 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046778976999999999, 2024-05-08T03:20:15.155 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27764 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179381+0000", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000122717 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179384+0000", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5000000000000002e-06 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179389+0000", 2024-05-08T03:20:15.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179392+0000", 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9280000000000003e-06 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225880+0000", 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046487354000000002 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225944+0000", 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4657000000000001e-05 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226038+0000", 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3066999999999995e-05 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27765 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 802816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.920699639, 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046843500000000003, 2024-05-08T03:20:15.159 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27765 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181093+0000", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016927820000000001 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181096+0000", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.506e-06 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181103+0000", 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181108+0000", 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5979999999999996e-06 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226080+0000", 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044971481000000001 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226152+0000", 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2057000000000005e-05 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226244+0000", 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1891000000000005e-05 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27776 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.918763099, 2024-05-08T03:20:15.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046303351999999999, 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27776 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2599999999999998e-07 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181338+0000", 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.266e-06 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181341+0000", 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181344+0000", 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227544+0000", 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046199656999999998 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227585+0000", 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1850000000000001e-05 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227640+0000", 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4586000000000002e-05 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27777 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:15.167 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.918752904, 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046409785000000002, 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27777 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181348+0000", 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.145e-06 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181351+0000", 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8210000000000002e-06 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181354+0000", 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5900000000000002e-06 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227662+0000", 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046308716 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227703+0000", 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0141000000000001e-05 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227757+0000", 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4147000000000002e-05 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27778 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:15.171 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.918743183, 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046521521000000003, 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27778 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181358+0000", 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.167e-06 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181362+0000", 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181364+0000", 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6019999999999998e-06 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227780+0000", 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046415911999999997 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227821+0000", 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1179e-05 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227878+0000", 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6736e-05 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27779 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.175 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.918732649000001, 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046626723000000002, 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27779 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183095+0000", 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001728282 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183102+0000", 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4790000000000001e-06 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183117+0000", 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4929999999999999e-05 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183126+0000", 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9209999999999998e-06 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227901+0000", 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044775342000000003 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227940+0000", 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9332999999999998e-05 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227994+0000", 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3436000000000001e-05 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27794 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1269760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.179 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.916593947999999, 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046263348000000003, 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27794 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183508+0000", 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.057e-06 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183514+0000", 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6710000000000004e-06 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183519+0000", 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8269999999999997e-06 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229676+0000", 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046157090999999997 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229716+0000", 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9870000000000003e-05 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229769+0000", 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3477000000000003e-05 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27795 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1294336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.183 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.916575084, 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046360547000000002, 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27795 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:15.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183527+0000", 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:20:15.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183533+0000", 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183537+0000", 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4660000000000001e-06 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229792+0000", 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046254587999999999 2024-05-08T03:20:15.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229832+0000", 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0055999999999997e-05 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229885+0000", 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3488e-05 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.187 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27796 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1441792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.916556635999999, 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046459223000000001, 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27796 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:15.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185095+0000", 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015521389999999999 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185101+0000", 2024-05-08T03:20:15.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185115+0000", 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3735e-05 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185123+0000", 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1359999999999997e-06 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229910+0000", 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044786908 2024-05-08T03:20:15.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229949+0000", 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9192e-05 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.230002+0000", 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3601999999999999e-05 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:15.191 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:15.192 INFO:teuthology.orchestra.run.smithi177.stdout:2827 write 0x9cce029 thru 0x9cd830d (0xa2e5 bytes) 2024-05-08T03:20:15.193 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:15.194 INFO:teuthology.orchestra.run.smithi177.stdout:2829 punch from 0x6adc493 to 0x6add35b, (0xec8 bytes) 2024-05-08T03:20:15.194 INFO:teuthology.orchestra.run.smithi177.stdout:2830 write 0x1d90725 thru 0x1d9de12 (0xd6ee bytes) 2024-05-08T03:20:15.194 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:15.194 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:15.195 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:15.195 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:15.200 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:15.191+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:15.215 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:15.216 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:15.255 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:15.255 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:15.293 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:15.291+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:15.296 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:15.296 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:15.296 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:15.327 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:15.327 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:15.354 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:15.354 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:15.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:15.359 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:15.360 INFO:teuthology.orchestra.run.smithi177.stdout:2831 trunc from 0x9cd830e to 0x8afff57 2024-05-08T03:20:15.371 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.379 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:15.379 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:15.379 INFO:teuthology.orchestra.run.smithi177.stdout:2833 trunc from 0x8afff57 to 0xe80babe 2024-05-08T03:20:15.385 INFO:teuthology.orchestra.run.smithi177.stdout:2834 punch from 0x4840dc1 to 0x484ed2f, (0xdf6e bytes) 2024-05-08T03:20:15.394 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:15.395 INFO:teuthology.orchestra.run.smithi177.stdout:2837 trunc from 0xe80babe to 0x7b85d59 2024-05-08T03:20:15.395 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:15.392+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:15.407 INFO:teuthology.orchestra.run.smithi177.stdout:2839 trunc from 0x7b85d59 to 0x173b7aa 2024-05-08T03:20:15.494 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:15.493+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:15.571 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:15.572 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:15.583 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:15.606 INFO:teuthology.orchestra.run.smithi177.stdout:2840 read 0x29ae78 thru 0x29c1ae (0x1337 bytes) 2024-05-08T03:20:15.607 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:15.604+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:15.607 INFO:teuthology.orchestra.run.smithi177.stdout:2841 read 0x2366b2 thru 0x245ab7 (0xf406 bytes) 2024-05-08T03:20:15.616 INFO:teuthology.orchestra.run.smithi177.stdout:2842 trunc from 0x173b7aa to 0x8cc3a4a 2024-05-08T03:20:15.621 INFO:teuthology.orchestra.run.smithi177.stdout:2844 write 0x4fec49b thru 0x4ff9861 (0xd3c7 bytes) 2024-05-08T03:20:15.633 INFO:teuthology.orchestra.run.smithi177.stdout:2845 read 0x841b3b thru 0x84b8d9 (0x9d9f bytes) 2024-05-08T03:20:15.635 INFO:teuthology.orchestra.run.smithi177.stdout:2846 punch from 0x35a7e93 to 0x35b34b5, (0xb622 bytes) 2024-05-08T03:20:15.635 INFO:teuthology.orchestra.run.smithi177.stdout:2847 read 0x2df7845 thru 0x2dfc84e (0x500a bytes) 2024-05-08T03:20:15.635 INFO:teuthology.orchestra.run.smithi177.stdout:2848 read 0x6728c4b thru 0x67379bf (0xed75 bytes) 2024-05-08T03:20:15.635 INFO:teuthology.orchestra.run.smithi177.stdout:2851 read 0x50adae5 thru 0x50b95c2 (0xbade bytes) 2024-05-08T03:20:15.636 INFO:teuthology.orchestra.run.smithi177.stdout:2852 punch from 0x672eb63 to 0x6732adf, (0x3f7c bytes) 2024-05-08T03:20:15.636 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.638 INFO:teuthology.orchestra.run.smithi177.stdout:2855 trunc from 0x8cc3a4a to 0xb28460c 2024-05-08T03:20:15.643 INFO:teuthology.orchestra.run.smithi177.stdout:2856 write 0xa40557d thru 0xa408654 (0x30d8 bytes) 2024-05-08T03:20:15.649 INFO:teuthology.orchestra.run.smithi177.stdout:2857 writesame 0xab4e70c thru 0xab53bf7 data_size 0xa(0x54ec bytes) 2024-05-08T03:20:15.654 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.667 INFO:teuthology.orchestra.run.smithi177.stdout:2859 trunc from 0xb28460c to 0x2b8b8d 2024-05-08T03:20:15.696 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:15.694+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:15.756 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:15.756 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:15.768 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:15.810 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:15.808+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:15.821 INFO:teuthology.orchestra.run.smithi177.stdout:2860 punch from 0xbee0b to 0xc2d2d, (0x3f22 bytes) 2024-05-08T03:20:15.826 INFO:teuthology.orchestra.run.smithi177.stdout:2861 write 0x38c9a3 thru 0x38d3e9 (0xa47 bytes) 2024-05-08T03:20:15.841 INFO:teuthology.orchestra.run.smithi177.stdout:2862 write 0x70602fd thru 0x706de67 (0xdb6b bytes) 2024-05-08T03:20:15.851 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.851 INFO:teuthology.orchestra.run.smithi177.stdout:2866 write 0x1b6e442 thru 0x1b7c76b (0xe32a bytes) 2024-05-08T03:20:15.863 INFO:teuthology.orchestra.run.smithi177.stdout:2867 punch from 0x27ddb6a to 0x27e0207, (0x269d bytes) 2024-05-08T03:20:15.863 INFO:teuthology.orchestra.run.smithi177.stdout:2871 read 0x5bca279 thru 0x5bd43cc (0xa154 bytes) 2024-05-08T03:20:15.867 INFO:teuthology.orchestra.run.smithi177.stdout:2872 trunc from 0x706de68 to 0x323dc14 2024-05-08T03:20:15.886 INFO:teuthology.orchestra.run.smithi177.stdout:2874 write 0x652794c thru 0x65347d5 (0xce8a bytes) 2024-05-08T03:20:15.897 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:15.896+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:15.905 INFO:teuthology.orchestra.run.smithi177.stdout:2875 trunc from 0x65347d6 to 0xb28b02e 2024-05-08T03:20:15.910 INFO:teuthology.orchestra.run.smithi177.stdout:2876 write 0x7ebe8d7 thru 0x7ec61d5 (0x78ff bytes) 2024-05-08T03:20:15.918 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.918 INFO:teuthology.orchestra.run.smithi177.stdout:2879 read 0x312b313 thru 0x312d264 (0x1f52 bytes) 2024-05-08T03:20:15.925 INFO:teuthology.orchestra.run.smithi177.stdout:2880 writesame 0xc6daff8 thru 0xc6e0f14 data_size 0x1(0x5f1d bytes) 2024-05-08T03:20:15.932 INFO:teuthology.orchestra.run.smithi177.stdout:2883 write 0xd9d9f55 thru 0xd9e086c (0x6918 bytes) 2024-05-08T03:20:15.936 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:15.937 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:15.939 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.939 INFO:teuthology.orchestra.run.smithi177.stdout:2885 write 0x5ca4f43 thru 0x5ca92a2 (0x4360 bytes) 2024-05-08T03:20:15.948 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:15.948 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:15.959 INFO:teuthology.orchestra.run.smithi177.stdout:2887 trunc from 0xd9e086d to 0x378ebd0 2024-05-08T03:20:15.986 INFO:teuthology.orchestra.run.smithi177.stdout:2889 trunc from 0x378ebd0 to 0xdf6bb58 2024-05-08T03:20:15.998 INFO:teuthology.orchestra.run.smithi177.stdout:2890 trunc from 0xdf6bb58 to 0x8601f15 2024-05-08T03:20:15.999 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:15.996+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:16.004 INFO:teuthology.orchestra.run.smithi177.stdout:2891 trunc from 0x8601f15 to 0x87e4181 2024-05-08T03:20:16.008 INFO:teuthology.orchestra.run.smithi177.stdout:2894 punch from 0x60bcabc to 0x60c65ac, (0x9af0 bytes) 2024-05-08T03:20:16.015 INFO:teuthology.orchestra.run.smithi177.stdout:2895 write 0x952b02d thru 0x952dbe9 (0x2bbd bytes) 2024-05-08T03:20:16.020 INFO:teuthology.orchestra.run.smithi177.stdout:2896 writesame 0x886336f thru 0x887213d data_size 0x27(0xedcf bytes) 2024-05-08T03:20:16.042 INFO:teuthology.orchestra.run.smithi177.stdout:2897 writesame 0xc522ab0 thru 0xc52f1a9 data_size 0x1(0xc6fa bytes) 2024-05-08T03:20:16.042 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:16.042 INFO:teuthology.orchestra.run.smithi177.stdout:2899 punch from 0x1c19e4b to 0x1c26f12, (0xd0c7 bytes) 2024-05-08T03:20:16.051 INFO:teuthology.orchestra.run.smithi177.stdout:2900 trunc from 0xc52f1aa to 0x5145186 2024-05-08T03:20:16.063 INFO:teuthology.orchestra.run.smithi177.stdout:2901 write 0x166e054 thru 0x16779f9 (0x99a6 bytes) 2024-05-08T03:20:16.084 INFO:teuthology.orchestra.run.smithi177.stdout:2903 trunc from 0x5145186 to 0xa6478e4 2024-05-08T03:20:16.086 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.086 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.093 INFO:teuthology.orchestra.run.smithi177.stderr:set 1.5 pg_upmap mapping to [2,0] 2024-05-08T03:20:16.097 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:20:16.099 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:16.107 INFO:teuthology.orchestra.run.smithi177.stdout:2904 writesame 0xd79668d thru 0xd79af49 data_size 0x3(0x48bd bytes) 2024-05-08T03:20:16.108 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:16.107+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:16.114 INFO:teuthology.orchestra.run.smithi177.stdout:2905 write 0xe40ecf8 thru 0xe4177ed (0x8af6 bytes) 2024-05-08T03:20:16.126 INFO:teuthology.orchestra.run.smithi177.stdout:2906 read 0x46e3b7 thru 0x47573e (0x7388 bytes) 2024-05-08T03:20:16.127 INFO:teuthology.orchestra.run.smithi177.stdout:2907 write 0x50f64c0 thru 0x50ff7b7 (0x92f8 bytes) 2024-05-08T03:20:16.136 INFO:teuthology.orchestra.run.smithi177.stdout:2910 writesame 0x4b76334 thru 0x4b81647 data_size 0x4(0xb314 bytes) 2024-05-08T03:20:16.142 INFO:teuthology.orchestra.run.smithi177.stdout:2911 write 0x53abc84 thru 0x53b96c1 (0xda3e bytes) 2024-05-08T03:20:16.145 INFO:teuthology.orchestra.run.smithi177.stdout:2913 read 0x981e4e thru 0x98ca55 (0xac08 bytes) 2024-05-08T03:20:16.146 INFO:teuthology.orchestra.run.smithi177.stdout:2916 read 0x8b90ab0 thru 0x8b9d224 (0xc775 bytes) 2024-05-08T03:20:16.146 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:16.146 INFO:teuthology.orchestra.run.smithi177.stdout:2919 write 0xe044b3c thru 0xe046835 (0x1cfa bytes) 2024-05-08T03:20:16.150 INFO:teuthology.orchestra.run.smithi177.stdout:2921 trunc from 0xe4177ee to 0xe250b28 2024-05-08T03:20:16.177 INFO:teuthology.orchestra.run.smithi177.stdout:2922 trunc from 0xe250b28 to 0x3e8ce86 2024-05-08T03:20:16.188 INFO:teuthology.orchestra.run.smithi177.stdout:2923 punch from 0x38158e7 to 0x3816a8c, (0x11a5 bytes) 2024-05-08T03:20:16.200 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:16.199+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:16.203 INFO:teuthology.orchestra.run.smithi177.stdout:2924 write 0xb1006ed thru 0xb10af7b (0xa88f bytes) 2024-05-08T03:20:16.219 INFO:teuthology.orchestra.run.smithi177.stdout:2925 writesame 0xbed6611 thru 0xbed925b data_size 0x17(0x2c4b bytes) 2024-05-08T03:20:16.219 INFO:teuthology.orchestra.run.smithi177.stdout:2926 write 0x8bdddab thru 0x8be9f91 (0xc1e7 bytes) 2024-05-08T03:20:16.235 INFO:teuthology.orchestra.run.smithi177.stdout:2927 writesame 0xdcc67a2 thru 0xdcc80e0 data_size 0x1(0x193f bytes) 2024-05-08T03:20:16.235 INFO:teuthology.orchestra.run.smithi177.stdout:2929 read 0x56527bb thru 0x5654e79 (0x26bf bytes) 2024-05-08T03:20:16.235 INFO:teuthology.orchestra.run.smithi177.stdout:2931 read 0x34f87ef thru 0x350825d (0xfa6f bytes) 2024-05-08T03:20:16.235 INFO:teuthology.orchestra.run.smithi177.stdout:2932 write 0x1e4316 thru 0x1e49d9 (0x6c4 bytes) 2024-05-08T03:20:16.236 INFO:teuthology.orchestra.run.smithi177.stdout:2934 read 0xbc7327 thru 0xbcdf37 (0x6c11 bytes) 2024-05-08T03:20:16.237 INFO:teuthology.orchestra.run.smithi177.stdout:2935 read 0x4eaf776 thru 0x4eb682e (0x70b9 bytes) 2024-05-08T03:20:16.237 INFO:teuthology.orchestra.run.smithi177.stdout:2936 punch from 0xb8a9151 to 0xb8b6fc2, (0xde71 bytes) 2024-05-08T03:20:16.237 INFO:teuthology.orchestra.run.smithi177.stdout:2941 punch from 0x69f86ab to 0x69fbc0a, (0x355f bytes) 2024-05-08T03:20:16.238 INFO:teuthology.orchestra.run.smithi177.stdout:2944 writesame 0x899d2bf thru 0x89a80d7 data_size 0x1(0xae19 bytes) 2024-05-08T03:20:16.248 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.248 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.248 INFO:teuthology.orchestra.run.smithi177.stdout:2945 write 0x55cdc2e thru 0x55dc4d8 (0xe8ab bytes) 2024-05-08T03:20:16.260 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:16.262 INFO:teuthology.orchestra.run.smithi177.stdout:2946 write 0xd400abd thru 0xd40f0cd (0xe611 bytes) 2024-05-08T03:20:16.273 INFO:teuthology.orchestra.run.smithi177.stdout:2947 write 0xa5f51bf thru 0xa5fd0b0 (0x7ef2 bytes) 2024-05-08T03:20:16.284 INFO:teuthology.orchestra.run.smithi177.stdout:2948 read 0x12d7624 thru 0x12e6e90 (0xf86d bytes) 2024-05-08T03:20:16.291 INFO:teuthology.orchestra.run.smithi177.stdout:2949 write 0xedea64a thru 0xedecd98 (0x274f bytes) 2024-05-08T03:20:16.295 INFO:teuthology.orchestra.run.smithi177.stdout:2950 punch from 0x7c9f24 to 0x7d0f71, (0x704d bytes) 2024-05-08T03:20:16.295 INFO:teuthology.orchestra.run.smithi177.stdout:2951 write 0xe5ce728 thru 0xe5dd525 (0xedfe bytes) 2024-05-08T03:20:16.303 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:16.301+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:16.320 INFO:teuthology.orchestra.run.smithi177.stdout:2952 trunc from 0xedecd99 to 0x35d222c 2024-05-08T03:20:16.354 INFO:teuthology.orchestra.run.smithi177.stdout:2953 trunc from 0x35d222c to 0x86a73f 2024-05-08T03:20:16.370 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.371 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.374 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:16.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:16.392 INFO:teuthology.orchestra.run.smithi177.stdout:2955 write 0xacc9580 thru 0xacd616f (0xcbf0 bytes) 2024-05-08T03:20:16.403 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:16.402+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:16.405 INFO:teuthology.orchestra.run.smithi177.stdout:2956 trunc from 0xacd6170 to 0xcedcf8d 2024-05-08T03:20:16.410 INFO:teuthology.orchestra.run.smithi177.stdout:2957 writesame 0xcba69be thru 0xcbb1eb7 data_size 0xa(0xb4fa bytes) 2024-05-08T03:20:16.426 INFO:teuthology.orchestra.run.smithi177.stdout:2959 punch from 0xa0015dd to 0xa007e8e, (0x68b1 bytes) 2024-05-08T03:20:16.426 INFO:teuthology.orchestra.run.smithi177.stdout:2960 punch from 0xc37e63e to 0xc37e818, (0x1da bytes) 2024-05-08T03:20:16.426 INFO:teuthology.orchestra.run.smithi177.stdout:2961 writesame 0xacbfde thru 0xacdce8 data_size 0x1(0x1d0b bytes) 2024-05-08T03:20:16.430 INFO:teuthology.orchestra.run.smithi177.stdout:2962 clone 25 order 22 su 16384 sc 3 2024-05-08T03:20:16.489 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.489 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:16.505 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:16.504+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:16.604 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.604 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.605 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:16.604+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:16.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:16.707 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:16.705+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:16.759 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.759 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:16.809 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:16.807+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:16.909 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:16.908+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:16.912 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:16.912 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:16.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:17.010 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:17.008+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:17.028 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:17.028 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:17.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:17.096 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:20:17.111 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:17.109+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:17.152 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone24 from 0xcedcf8d (overlap 0x2b8b8d) to 0x0 2024-05-08T03:20:17.186 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:17.186 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:17.196 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:17.221 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-05-08T03:20:17.239 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:17.237+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:17.301 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:17.301 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:17.310 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:17.313 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:17.312+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:17.340 INFO:teuthology.orchestra.run.smithi177.stdout:2964 read 0x5b1efc5 thru 0x5b2e9f5 (0xfa31 bytes) 2024-05-08T03:20:17.340 INFO:teuthology.orchestra.run.smithi177.stdout:2965 write 0x7497768 thru 0x749a99f (0x3238 bytes) 2024-05-08T03:20:17.351 INFO:teuthology.orchestra.run.smithi177.stdout:2966 read 0xc74020c thru 0xc746f51 (0x6d46 bytes) 2024-05-08T03:20:17.352 INFO:teuthology.orchestra.run.smithi177.stdout:2968 write 0x6e91dc4 thru 0x6ea0481 (0xe6be bytes) 2024-05-08T03:20:17.355 INFO:teuthology.orchestra.run.smithi177.stdout:2970 writesame 0x3c8760b thru 0x3c93d01 data_size 0x3d(0xc6f7 bytes) 2024-05-08T03:20:17.359 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:17.368 INFO:teuthology.orchestra.run.smithi177.stdout:2971 read 0xb1d9931 thru 0xb1dc8dc (0x2fac bytes) 2024-05-08T03:20:17.369 INFO:teuthology.orchestra.run.smithi177.stdout:2972 read 0x2b42c3d thru 0x2b4c7f5 (0x9bb9 bytes) 2024-05-08T03:20:17.369 INFO:teuthology.orchestra.run.smithi177.stdout:2973 read 0x850ba1b thru 0x851b12e (0xf714 bytes) 2024-05-08T03:20:17.369 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.381 INFO:teuthology.orchestra.run.smithi177.stdout:2977 trunc from 0xcedcf8d to 0x286a48f 2024-05-08T03:20:17.398 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.398 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.398 INFO:teuthology.orchestra.run.smithi177.stdout:2981 punch from 0x98cd14 to 0x994f50, (0x823c bytes) 2024-05-08T03:20:17.407 INFO:teuthology.orchestra.run.smithi177.stdout:2982 trunc from 0x286a48f to 0xa1e3ac8 2024-05-08T03:20:17.413 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:17.412+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:17.420 INFO:teuthology.orchestra.run.smithi177.stdout:2983 trunc from 0xa1e3ac8 to 0x468b105 2024-05-08T03:20:17.425 INFO:teuthology.orchestra.run.smithi177.stdout:2984 read 0x1c777fc thru 0x1c7f50b (0x7d10 bytes) 2024-05-08T03:20:17.433 INFO:teuthology.orchestra.run.smithi177.stdout:2985 write 0x698946f thru 0x698cad7 (0x3669 bytes) 2024-05-08T03:20:17.453 INFO:teuthology.orchestra.run.smithi177.stdout:2986 write 0xea9de06 thru 0xeaabd95 (0xdf90 bytes) 2024-05-08T03:20:17.458 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:17.458 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:17.458 INFO:teuthology.orchestra.run.smithi177.stdout:2988 punch from 0x638ac0f to 0x638e352, (0x3743 bytes) 2024-05-08T03:20:17.458 INFO:teuthology.orchestra.run.smithi177.stdout:2989 punch from 0xbbd5674 to 0xbbe0f75, (0xb901 bytes) 2024-05-08T03:20:17.458 INFO:teuthology.orchestra.run.smithi177.stdout:2992 write 0x825007a thru 0x825e42e (0xe3b5 bytes) 2024-05-08T03:20:17.465 INFO:teuthology.orchestra.run.smithi177.stdout:2993 read 0xb947d24 thru 0xb95672f (0xea0c bytes) 2024-05-08T03:20:17.465 INFO:teuthology.orchestra.run.smithi177.stdout:2994 punch from 0x80918b8 to 0x809e6b8, (0xce00 bytes) 2024-05-08T03:20:17.465 INFO:teuthology.orchestra.run.smithi177.stdout:2995 punch from 0x61df87c to 0x61e78e3, (0x8067 bytes) 2024-05-08T03:20:17.465 INFO:teuthology.orchestra.run.smithi177.stdout:2996 write 0x2462648 thru 0x2470726 (0xe0df bytes) 2024-05-08T03:20:17.467 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:17.471 INFO:teuthology.orchestra.run.smithi177.stdout:2997 punch from 0x4c87de3 to 0x4c89061, (0x127e bytes) 2024-05-08T03:20:17.471 INFO:teuthology.orchestra.run.smithi177.stdout:2999 read 0x2c34d7b thru 0x2c41200 (0xc486 bytes) 2024-05-08T03:20:17.471 INFO:teuthology.orchestra.run.smithi177.stdout:3000 write 0xaadcddb thru 0xaaebd56 (0xef7c bytes) 2024-05-08T03:20:17.478 INFO:teuthology.orchestra.run.smithi177.stdout:3001 write 0x49c755b thru 0x49c9e36 (0x28dc bytes) 2024-05-08T03:20:17.484 INFO:teuthology.orchestra.run.smithi177.stdout:3003 read 0xb62f2f4 thru 0xb634928 (0x5635 bytes) 2024-05-08T03:20:17.484 INFO:teuthology.orchestra.run.smithi177.stdout:3004 read 0xcff8fc6 thru 0xcffd60b (0x4646 bytes) 2024-05-08T03:20:17.489 INFO:teuthology.orchestra.run.smithi177.stdout:3005 trunc from 0xeaabd96 to 0xadca762 2024-05-08T03:20:17.508 INFO:teuthology.orchestra.run.smithi177.stdout:3006 write 0x9058e7e thru 0x9066ee4 (0xe067 bytes) 2024-05-08T03:20:17.513 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:17.513 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:17.514 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:17.512+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:17.515 INFO:teuthology.orchestra.run.smithi177.stdout:3008 writesame 0x7964cb2 thru 0x79719b9 data_size 0xa2(0xcd08 bytes) 2024-05-08T03:20:17.527 INFO:teuthology.orchestra.run.smithi177.stdout:3009 write 0xd0ccd22 thru 0xd0d1922 (0x4c01 bytes) 2024-05-08T03:20:17.538 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:17.539 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:17.539 INFO:teuthology.orchestra.run.smithi177.stdout:3010 trunc from 0xd0d1923 to 0x7d6f8da 2024-05-08T03:20:17.552 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.558 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:17.559 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:17.560 INFO:teuthology.orchestra.run.smithi177.stdout:3012 write 0x96961ac thru 0x969b880 (0x56d5 bytes) 2024-05-08T03:20:17.564 INFO:teuthology.orchestra.run.smithi177.stdout:3014 read 0x9045337 thru 0x9050ef9 (0xbbc3 bytes) 2024-05-08T03:20:17.571 INFO:teuthology.orchestra.run.smithi177.stdout:3015 write 0x9dfb4ee thru 0x9e02bc0 (0x76d3 bytes) 2024-05-08T03:20:17.582 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:17.582 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:17.589 INFO:teuthology.orchestra.run.smithi177.stdout:3016 trunc from 0x9e02bc1 to 0x1576d6d 2024-05-08T03:20:17.602 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:17.602 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:17.602 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:17.602 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:17.606 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:17.606 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:17.607 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:17.613 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:17.615 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:17.613+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:17.625 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:17.626 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:17.639 INFO:teuthology.orchestra.run.smithi177.stdout:3018 write 0x8455add thru 0x84630c2 (0xd5e6 bytes) 2024-05-08T03:20:17.651 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:17.651 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:17.652 INFO:teuthology.orchestra.run.smithi177.stdout:3019 write 0xa2ec3a6 thru 0xa2fb53b (0xf196 bytes) 2024-05-08T03:20:17.655 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:17.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:17.673 INFO:teuthology.orchestra.run.smithi177.stdout:3020 writesame 0xbcf5051 thru 0xbd00e18 data_size 0x4(0xbdc8 bytes) 2024-05-08T03:20:17.673 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.673 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.673 INFO:teuthology.orchestra.run.smithi177.stdout:3024 punch from 0x24decdf to 0x24ead54, (0xc075 bytes) 2024-05-08T03:20:17.673 INFO:teuthology.orchestra.run.smithi177.stdout:3025 read 0x3caa81e thru 0x3cb628c (0xba6f bytes) 2024-05-08T03:20:17.673 INFO:teuthology.orchestra.run.smithi177.stdout:3027 punch from 0x65649c8 to 0x656a734, (0x5d6c bytes) 2024-05-08T03:20:17.685 INFO:teuthology.orchestra.run.smithi177.stdout:3028 trunc from 0xbd00e19 to 0xe8f251 2024-05-08T03:20:17.697 INFO:teuthology.orchestra.run.smithi177.stdout:3029 punch from 0x641b31 to 0x647a8c, (0x5f5b bytes) 2024-05-08T03:20:17.712 INFO:teuthology.orchestra.run.smithi177.stdout:3030 write 0x8962ed3 thru 0x896543b (0x2569 bytes) 2024-05-08T03:20:17.716 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:17.714+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:17.718 INFO:teuthology.orchestra.run.smithi177.stdout:3031 punch from 0x4107973 to 0x410d109, (0x5796 bytes) 2024-05-08T03:20:17.718 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.728 INFO:teuthology.orchestra.run.smithi177.stdout:3033 trunc from 0x896543c to 0x34a7d3 2024-05-08T03:20:17.756 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:17.756 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:17.756 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:17.756 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:17.756 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:17.757 INFO:teuthology.orchestra.run.smithi177.stdout:3034 punch from 0xaec21 to 0xbb10d, (0xc4ec bytes) 2024-05-08T03:20:17.757 INFO:teuthology.orchestra.run.smithi177.stdout:3035 read 0x2d9a86 thru 0x2e852d (0xeaa8 bytes) 2024-05-08T03:20:17.773 INFO:teuthology.orchestra.run.smithi177.stdout:3036 writesame 0x6946438 thru 0x694eb86 data_size 0x2f(0x874f bytes) 2024-05-08T03:20:17.773 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.774 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:17.784 INFO:teuthology.orchestra.run.smithi177.stdout:3038 write 0xc39f9b0 thru 0xc3a7168 (0x77b9 bytes) 2024-05-08T03:20:17.794 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.803 INFO:teuthology.orchestra.run.smithi177.stdout:3041 write 0xe54ad3f thru 0xe54d607 (0x28c9 bytes) 2024-05-08T03:20:17.808 INFO:teuthology.orchestra.run.smithi177.stdout:3042 write 0xa0b05fa thru 0xa0b5db6 (0x57bd bytes) 2024-05-08T03:20:17.812 INFO:teuthology.orchestra.run.smithi177.stdout:3043 read 0x89906d8 thru 0x8994a28 (0x4351 bytes) 2024-05-08T03:20:17.812 INFO:teuthology.orchestra.run.smithi177.stdout:3048 writesame 0x4d82deb thru 0x4d8d8f1 data_size 0x1(0xab07 bytes) 2024-05-08T03:20:17.817 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:17.815+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:17.827 INFO:teuthology.orchestra.run.smithi177.stdout:3049 write 0x1e11f8e thru 0x1e19179 (0x71ec bytes) 2024-05-08T03:20:17.835 INFO:teuthology.orchestra.run.smithi177.stdout:3050 punch from 0xc8e2dad to 0xc8e6820, (0x3a73 bytes) 2024-05-08T03:20:17.836 INFO:teuthology.orchestra.run.smithi177.stdout:3051 punch from 0x8c530c2 to 0x8c536eb, (0x629 bytes) 2024-05-08T03:20:17.836 INFO:teuthology.orchestra.run.smithi177.stdout:3052 writesame 0x416449e thru 0x41646a8 data_size 0x1(0x20b bytes) 2024-05-08T03:20:17.838 INFO:teuthology.orchestra.run.smithi177.stdout:3053 write 0x29506fe thru 0x2954e29 (0x472c bytes) 2024-05-08T03:20:17.844 INFO:teuthology.orchestra.run.smithi177.stdout:3054 write 0x5711f92 thru 0x5716a5e (0x4acd bytes) 2024-05-08T03:20:17.852 INFO:teuthology.orchestra.run.smithi177.stdout:3055 punch from 0x3771fe2 to 0x37787ed, (0x680b bytes) 2024-05-08T03:20:17.852 INFO:teuthology.orchestra.run.smithi177.stdout:3056 read 0x8aac005 thru 0x8aba57d (0xe579 bytes) 2024-05-08T03:20:17.852 INFO:teuthology.orchestra.run.smithi177.stdout:3059 writesame 0x466e977 thru 0x46731ad data_size 0x1(0x4837 bytes) 2024-05-08T03:20:17.865 INFO:teuthology.orchestra.run.smithi177.stdout:3060 write 0x7ec2539 thru 0x7ec43bc (0x1e84 bytes) 2024-05-08T03:20:17.870 INFO:teuthology.orchestra.run.smithi177.stdout:3061 punch from 0x917a5f9 to 0x91894d0, (0xeed7 bytes) 2024-05-08T03:20:17.870 INFO:teuthology.orchestra.run.smithi177.stdout:3062 read 0x86dc395 thru 0x86e159b (0x5207 bytes) 2024-05-08T03:20:17.870 INFO:teuthology.orchestra.run.smithi177.stdout:3063 write 0x61b18e5 thru 0x61b4068 (0x2784 bytes) 2024-05-08T03:20:17.875 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:17.875 INFO:teuthology.orchestra.run.smithi177.stdout:3065 read 0x1e49962 thru 0x1e4bf15 (0x25b4 bytes) 2024-05-08T03:20:17.875 INFO:teuthology.orchestra.run.smithi177.stdout:3066 clone 26 order 18 su 131072 sc 11 2024-05-08T03:20:17.905 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:17.905 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:17.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.703643651, 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:17.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:17.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:17.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.703634201, 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:17.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:17.913 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.703621464999998, 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.914 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:17.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.917 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.703610888999997, 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.918 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.703601501999998, 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.922 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:17.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.703592321999999, 2024-05-08T03:20:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472767287000003, 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472735049999997, 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472721720000003, 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.942 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472706746, 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.946 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472695436999999, 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:17.950 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472677767999997, 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472662569999997, 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.955 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:17.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 49.472648476000003, 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:17.959 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:17.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.963 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.697694503000001, 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:17.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:17.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:17.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:17.968 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 40.774800700999997, 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:17.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:17.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 40.774787213000003, 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.973 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:17.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 40.774758232000003, 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.977 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:17.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:17.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.980 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 40.774731625999998, 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.981 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:17.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.984 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 40.341983364999997, 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:17.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:17.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:17.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:17.990 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:17.991 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:17.992 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:17.990+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:18.092 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:18.091+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:18.149 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:18.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:18.149 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:18.149 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:18.159 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:18.160 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone25 from 0xe54d608 (overlap 0x34a7d3) to 0x2a1fdc 2024-05-08T03:20:18.204 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:18.203+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:18.247 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-05-08T03:20:18.248 INFO:teuthology.orchestra.run.smithi177.stdout:3067 read 0x3e94fbc thru 0x3e9e5df (0x9624 bytes) 2024-05-08T03:20:18.249 INFO:teuthology.orchestra.run.smithi177.stdout:3069 write 0xbe862c0 thru 0xbe907ed (0xa52e bytes) 2024-05-08T03:20:18.256 INFO:teuthology.orchestra.run.smithi177.stdout:3072 write 0xc862140 thru 0xc86a507 (0x83c8 bytes) 2024-05-08T03:20:18.269 INFO:teuthology.orchestra.run.smithi177.stdout:3073 trunc from 0xe54d608 to 0x671dd39 2024-05-08T03:20:18.292 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:18.292 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:18.292 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:18.292 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:18.292 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:18.294 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:18.294 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:18.292+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:18.303 INFO:teuthology.orchestra.run.smithi177.stdout:3076 write 0x9c2f3ff thru 0x9c3b22c (0xbe2e bytes) 2024-05-08T03:20:18.304 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:18.316 INFO:teuthology.orchestra.run.smithi177.stdout:3077 trunc from 0x9c3b22d to 0x375966c 2024-05-08T03:20:18.340 INFO:teuthology.orchestra.run.smithi177.stdout:3078 read 0x30e6f9b thru 0x30e8457 (0x14bd bytes) 2024-05-08T03:20:18.340 INFO:teuthology.orchestra.run.smithi177.stdout:3080 write 0x1cd94ca thru 0x1ce9127 (0xfc5e bytes) 2024-05-08T03:20:18.349 INFO:teuthology.orchestra.run.smithi177.stdout:3081 punch from 0x45cf20 to 0x465743, (0x8823 bytes) 2024-05-08T03:20:18.359 INFO:teuthology.orchestra.run.smithi177.stdout:3085 write 0x758c580 thru 0x7597f33 (0xb9b4 bytes) 2024-05-08T03:20:18.371 INFO:teuthology.orchestra.run.smithi177.stdout:3086 write 0xa49ed07 thru 0xa4a78c0 (0x8bba bytes) 2024-05-08T03:20:18.379 INFO:teuthology.orchestra.run.smithi177.stdout:3088 trunc from 0xa4a78c1 to 0xae7e015 2024-05-08T03:20:18.385 INFO:teuthology.orchestra.run.smithi177.stdout:3089 write 0x9722d95 thru 0x97323ab (0xf617 bytes) 2024-05-08T03:20:18.391 INFO:teuthology.orchestra.run.smithi177.stdout:3090 trunc from 0xae7e015 to 0xc9e35ba 2024-05-08T03:20:18.394 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:18.393+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:18.403 INFO:teuthology.orchestra.run.smithi177.stdout:3091 read 0x9306f32 thru 0x930f4db (0x85aa bytes) 2024-05-08T03:20:18.403 INFO:teuthology.orchestra.run.smithi177.stdout:3092 read 0x4dc27a3 thru 0x4dc4b50 (0x23ae bytes) 2024-05-08T03:20:18.403 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:18.403 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:18.403 INFO:teuthology.orchestra.run.smithi177.stdout:3096 writesame 0xc78feb0 thru 0xc79b1b9 data_size 0x3(0xb30a bytes) 2024-05-08T03:20:18.415 INFO:teuthology.orchestra.run.smithi177.stdout:3097 trunc from 0xc9e35ba to 0xe18cf77 2024-05-08T03:20:18.433 INFO:teuthology.orchestra.run.smithi177.stdout:3098 trunc from 0xe18cf77 to 0x3ad3682 2024-05-08T03:20:18.440 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:18.440 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.653132427000003, 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.441 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:18.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 28.505898278, 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.315758111000001, 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:18.454 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:18.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.291017565000001, 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.460 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:18.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25865 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [sparse-read 0~19262493] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.969150365000001, 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053445962, 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25865 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.466 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469276+0000", 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469279+0000", 2024-05-08T03:20:18.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.852e-06 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469291+0000", 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2098e-05 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469313+0000", 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2657999999999999e-05 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469326+0000", 2024-05-08T03:20:18.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2782000000000001e-05 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469401+0000", 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4759999999999996e-05 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469495+0000", 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4000999999999997e-05 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.522722+0000", 2024-05-08T03:20:18.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.053226810999999999 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25866 3.1a 3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.968529125, 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049237975000000003, 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25866 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469897+0000", 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469898+0000", 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.23e-07 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469902+0000", 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4839999999999998e-06 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469913+0000", 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0652999999999999e-05 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.469926+0000", 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3010000000000001e-05 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470021+0000", 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5622e-05 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470105+0000", 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3938999999999997e-05 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.519135+0000", 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049029544000000001 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:25871 3.16 3:692ca2ad:::rbd_data.118227bb2f9.0000000000000006:head [sparse-read 0~16777216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.968342356000001, 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.047103610999999997, 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.474 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 25871 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:18.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470084+0000", 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.06e-07 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470092+0000", 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8310000000000001e-06 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470099+0000", 2024-05-08T03:20:18.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0999999999999998e-06 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470108+0000", 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3190000000000001e-06 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470162+0000", 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3618000000000001e-05 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.470234+0000", 2024-05-08T03:20:18.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2342999999999995e-05 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:03.517187+0000", 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046952894000000002 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27760 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 557056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.259205357000001, 2024-05-08T03:20:18.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046225914999999999, 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27760 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179221+0000", 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8900000000000001e-07 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179222+0000", 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.114e-06 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179225+0000", 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9189999999999999e-06 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179228+0000", 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.819e-06 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225336+0000", 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046108300999999997 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225388+0000", 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2057e-05 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225447+0000", 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8415999999999998e-05 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.482 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27761 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.259195458000001, 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046341465999999998, 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27761 2024-05-08T03:20:18.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179231+0000", 2024-05-08T03:20:18.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179232+0000", 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179235+0000", 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8770000000000002e-06 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179237+0000", 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3690000000000001e-06 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225475+0000", 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046237568 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225515+0000", 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0080000000000003e-05 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225572+0000", 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7351e-05 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27762 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 581632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.259186387, 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046463320000000002, 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27762 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179240+0000", 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.29e-07 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179241+0000", 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.263e-06 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179244+0000", 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7769999999999998e-06 2024-05-08T03:20:18.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179246+0000", 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5509999999999998e-06 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225609+0000", 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046362154000000003 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225650+0000", 2024-05-08T03:20:18.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225703+0000", 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3260999999999997e-05 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27763 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.259176809, 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046573208999999997, 2024-05-08T03:20:18.491 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27763 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179249+0000", 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179251+0000", 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.102e-06 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179253+0000", 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8229999999999999e-06 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179256+0000", 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4949999999999998e-06 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225727+0000", 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046471512999999999 2024-05-08T03:20:18.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225768+0000", 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0142999999999998e-05 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225822+0000", 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4926000000000002e-05 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.495 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27764 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.259167471, 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046778976999999999, 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27764 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179259+0000", 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179381+0000", 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000122717 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179384+0000", 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5000000000000002e-06 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179389+0000", 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7539999999999997e-06 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179392+0000", 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9280000000000003e-06 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225880+0000", 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046487354000000002 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.225944+0000", 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4657000000000001e-05 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226038+0000", 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3066999999999995e-05 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.499 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27765 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 802816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.259025931, 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046843500000000003, 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27765 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.500 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.179400+0000", 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181093+0000", 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0016927820000000001 2024-05-08T03:20:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181096+0000", 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.506e-06 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181103+0000", 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181108+0000", 2024-05-08T03:20:18.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5979999999999996e-06 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226080+0000", 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044971481000000001 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226152+0000", 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2057000000000005e-05 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.226244+0000", 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1891000000000005e-05 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27776 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.257089390999999, 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046303351999999999, 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.504 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27776 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181337+0000", 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2599999999999998e-07 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181338+0000", 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.266e-06 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181341+0000", 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:18.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181344+0000", 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.74e-06 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227544+0000", 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046199656999999998 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227585+0000", 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1850000000000001e-05 2024-05-08T03:20:18.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227640+0000", 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4586000000000002e-05 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27777 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.257079195999999, 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046409785000000002, 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.508 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27777 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181347+0000", 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181348+0000", 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.145e-06 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181351+0000", 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8210000000000002e-06 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181354+0000", 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5900000000000002e-06 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227662+0000", 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046308716 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227703+0000", 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0141000000000001e-05 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227757+0000", 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4147000000000002e-05 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27778 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:18.512 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.257069475, 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046521521000000003, 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27778 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181357+0000", 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181358+0000", 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.167e-06 2024-05-08T03:20:18.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181362+0000", 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181364+0000", 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6019999999999998e-06 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227780+0000", 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046415911999999997 2024-05-08T03:20:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227821+0000", 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1179e-05 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227878+0000", 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6736e-05 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27779 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.257058941, 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046626723000000002, 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27779 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.181367+0000", 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183095+0000", 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001728282 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183102+0000", 2024-05-08T03:20:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4790000000000001e-06 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183117+0000", 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4929999999999999e-05 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183126+0000", 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9209999999999998e-06 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227901+0000", 2024-05-08T03:20:18.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044775342000000003 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227940+0000", 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9332999999999998e-05 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.227994+0000", 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3436000000000001e-05 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27794 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1269760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.254920240000001, 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046263348000000003, 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27794 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183506+0000", 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183508+0000", 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.057e-06 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183514+0000", 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6710000000000004e-06 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183519+0000", 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8269999999999997e-06 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229676+0000", 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046157090999999997 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229716+0000", 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9870000000000003e-05 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229769+0000", 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3477000000000003e-05 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.524 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27795 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1294336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.254901375999999, 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046360547000000002, 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27795 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.525 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183525+0000", 2024-05-08T03:20:18.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8099999999999998e-07 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183527+0000", 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183533+0000", 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183537+0000", 2024-05-08T03:20:18.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4660000000000001e-06 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229792+0000", 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046254587999999999 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229832+0000", 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0055999999999997e-05 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229885+0000", 2024-05-08T03:20:18.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3488e-05 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:27796 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:13 [read 1441792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.254882928000001, 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.046459223000000001, 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.529 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 27796 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.183543+0000", 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185095+0000", 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015521389999999999 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185101+0000", 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185115+0000", 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3735e-05 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.185123+0000", 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1359999999999997e-06 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229910+0000", 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.044786908 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.229949+0000", 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9192e-05 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:04.230002+0000", 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3601999999999999e-05 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.533 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:18.534 INFO:teuthology.orchestra.run.smithi177.stdout:3099 read 0xabab53 thru 0xac39c1 (0x8e6f bytes) 2024-05-08T03:20:18.534 INFO:teuthology.orchestra.run.smithi177.stdout:3101 read 0x31793ea thru 0x3182935 (0x954c bytes) 2024-05-08T03:20:18.535 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3103 writesame 0x920a03d thru 0x9217f6c data_size 0x8(0xdf30 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3104 write 0xb145550 thru 0xb14da84 (0x8535 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3105 write 0x815d806 thru 0x81621ce (0x49c9 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3106 write 0xd07d46d thru 0xd07f60a (0x219e bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3107 write 0x5f16d29 thru 0x5f1b8c9 (0x4ba1 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3109 punch from 0x79e92a1 to 0x79eff30, (0x6c8f bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3110 punch from 0xaa491c to 0xaa719d, (0x2881 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3111 punch from 0x3cb668a to 0x3cb815e, (0x1ad4 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3112 write 0xa5ffa61 thru 0xa603bf3 (0x4193 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3113 read 0xa989254 thru 0xa9981fa (0xefa7 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3115 read 0xb305502 thru 0xb30ad46 (0x5845 bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3116 read 0x70b36db thru 0x70b8689 (0x4faf bytes) 2024-05-08T03:20:18.538 INFO:teuthology.orchestra.run.smithi177.stdout:3117 read 0x1a08efa thru 0x1a0c306 (0x340d bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3118 write 0x592ea33 thru 0x5933c6e (0x523c bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3119 punch from 0x71cb107 to 0x71cf30e, (0x4207 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3121 write 0x27661e2 thru 0x276680b (0x62a bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3122 write 0x5360184 thru 0x5366abc (0x6939 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3124 read 0x4ddf9b8 thru 0x4ded2af (0xd8f8 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3126 read 0x9c76c9b thru 0x9c837a4 (0xcb0a bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3127 write 0xc7dc70a thru 0xc7df7a6 (0x309d bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3129 write 0xc87c29a thru 0xc88a44e (0xe1b5 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3133 read 0x3d433d4 thru 0x3d50414 (0xd041 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3135 write 0x8ce2fa0 thru 0x8ce77d4 (0x4835 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3136 read 0x4447f90 thru 0x44568e4 (0xe955 bytes) 2024-05-08T03:20:18.539 INFO:teuthology.orchestra.run.smithi177.stdout:3137 clone 27 order 23 su 2097152 sc 6 2024-05-08T03:20:18.540 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:18.533+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:18.635 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:18.634+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:18.681 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:18.681 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:18.682 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:18.682 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:18.690 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:18.736 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:18.734+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:18.835 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:18.835 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:18.835 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:18.835 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:18.835 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:18.838 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:18.836+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:18.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:18.939 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:18.937+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:18.951 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.848974949, 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:18.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.847254479, 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:18.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:18.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.960 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.847221038000001, 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:18.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.845795299000001, 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:18.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:18.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:18.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.969 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.845702684999999, 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:18.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:18.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.845192011, 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.978 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.845164502999999, 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:18.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:18.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:18.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:18.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.844758071999999, 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.844302694, 2024-05-08T03:20:18.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:18.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:18.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.843774446999999, 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:18.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:18.996 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.843301949000001, 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:18.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:18.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:18.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.843223817, 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.001 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:19.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.843148395, 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.006 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:19.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.843071232, 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.010 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:19.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:19.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.014 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.842993709, 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.015 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:19.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:19.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:19.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.842966797000001, 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:19.019 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:19.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:19.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:19.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.842945586999999, 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.028 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.842925807, 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:19.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.842371281, 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:19.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:19.037 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 10.842352268999999, 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:19.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:19.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:19.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:19.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:19.042 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.042 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.042 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.042 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.042 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.042 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:19.043 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:19.044 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:19.042+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:19.113 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:20:19.144 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:19.142+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:19.177 INFO:teuthology.orchestra.run.smithi177.stdout:leaving image image_client.0-clone26 intact 2024-05-08T03:20:19.233 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:19.233 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:19.233 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:19.233 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:19.246 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:19.245+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:19.250 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:19.347 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:19.345+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:19.388 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-05-08T03:20:19.397 INFO:teuthology.orchestra.run.smithi177.stdout:3138 write 0x26106b4 thru 0x261ac08 (0xa555 bytes) 2024-05-08T03:20:19.415 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:19.415 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:19.415 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:19.415 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:19.415 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:19.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:19.453 INFO:teuthology.orchestra.run.smithi177.stdout:3141 writesame 0xdb770c1 thru 0xdb7ec39 data_size 0x1(0x7b79 bytes) 2024-05-08T03:20:19.453 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:19.451+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:19.457 INFO:teuthology.orchestra.run.smithi177.stdout:3142 writesame 0xa9bb2ff thru 0xa9c58c0 data_size 0x31(0xa5c2 bytes) 2024-05-08T03:20:19.465 INFO:teuthology.orchestra.run.smithi177.stdout:3144 read 0x8ef4b83 thru 0x8ef5a4e (0xecc bytes) 2024-05-08T03:20:19.465 INFO:teuthology.orchestra.run.smithi177.stdout:3145 writesame 0x1831702 thru 0x183606d data_size 0x4a(0x496c bytes) 2024-05-08T03:20:19.513 INFO:teuthology.orchestra.run.smithi177.stdout:3147 trunc from 0xdb7ec3a to 0xc194a90 2024-05-08T03:20:19.537 INFO:teuthology.orchestra.run.smithi177.stdout:3149 write 0x3c41144 thru 0x3c4c70c (0xb5c9 bytes) 2024-05-08T03:20:19.542 INFO:teuthology.orchestra.run.smithi177.stdout:3150 read 0xe8f1c0 thru 0xe9e658 (0xf499 bytes) 2024-05-08T03:20:19.542 INFO:teuthology.orchestra.run.smithi177.stdout:3151 read 0x76723cf thru 0x767c339 (0x9f6b bytes) 2024-05-08T03:20:19.542 INFO:teuthology.orchestra.run.smithi177.stdout:3152 read 0x6c3cfd9 thru 0x6c44fbf (0x7fe7 bytes) 2024-05-08T03:20:19.548 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:19.546+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:19.556 INFO:teuthology.orchestra.run.smithi177.stdout:3153 trunc from 0xc194a90 to 0xa080c3 2024-05-08T03:20:19.590 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:19.590 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:19.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:19.590 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 20.587231277000001, 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.591 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:19.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:19.596 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 20.587163813, 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:19.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:19.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:19.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:19.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 20.527329480999999, 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.603 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:19.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:19.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 19.460821467999999, 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 19.434283822000001, 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.614 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:19.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:19.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 19.406372053999998, 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.623 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23281 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'92, mlcod=53'88)", 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 19.356826535, 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074279843999999998, 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:19.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226785+0000", 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.299e-06 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226794+0000", 2024-05-08T03:20:19.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5040000000000008e-06 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226810+0000", 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5781e-05 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226820+0000", 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9310000000000003e-06 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262249+0000", 2024-05-08T03:20:19.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.035428501000000001 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262269+0000", 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0077000000000001e-05 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301028+0000", 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.038759184000000002 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301064+0000", 2024-05-08T03:20:19.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5567e-05 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:50996 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.173304+0000", 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4103060269999999, 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069887774, 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.628 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 50996 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173304+0000", 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173304+0000", 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173305+0000", 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.37e-07 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173307+0000", 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0810000000000001e-06 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173313+0000", 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.348e-06 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173318+0000", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.242419+0000", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.06910057 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.242486+0000", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7497999999999996e-05 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243192+0000", 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070565000000000005 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:50997 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4102854689999997, 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070252423999999994, 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 50997 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8599999999999999e-07 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173327+0000", 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1019999999999999e-06 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173334+0000", 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6120000000000004e-06 2024-05-08T03:20:19.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173339+0000", 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2460000000000003e-06 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243218+0000", 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069878632999999996 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243278+0000", 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0443999999999999e-05 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243577+0000", 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00029900100000000002 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:50998 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.173345+0000", 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4102647489999995, 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070754912000000003, 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.636 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 50998 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173345+0000", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173345+0000", 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175140+0000", 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001794152 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175147+0000", 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2180000000000002e-06 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175158+0000", 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1331999999999999e-05 2024-05-08T03:20:19.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175165+0000", 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3390000000000004e-06 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243606+0000", 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.068440584999999998 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243678+0000", 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2213999999999997e-05 2024-05-08T03:20:19.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.244100+0000", 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042207200000000001 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51000 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4084083879999998, 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070034812000000002, 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51000 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:19.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175205+0000", 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.43e-06 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175212+0000", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1439999999999997e-06 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175217+0000", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2989999999999996e-06 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.244508+0000", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069290884999999997 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.244566+0000", 2024-05-08T03:20:19.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7807999999999999e-05 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.245237+0000", 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000670749 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51001 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 417792~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4083863900000004, 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071026390999999994, 2024-05-08T03:20:19.644 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51001 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0200000000000003e-07 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175226+0000", 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1859999999999999e-06 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175233+0000", 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4609999999999996e-06 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175238+0000", 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2379999999999997e-06 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.245267+0000", 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070028956000000003 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.245337+0000", 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0176e-05 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246250+0000", 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091297199999999996 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51002 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175244+0000", 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4083657819999997, 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071040220000000001, 2024-05-08T03:20:19.648 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51002 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175244+0000", 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175244+0000", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175245+0000", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4000000000000002e-07 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175247+0000", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2110000000000001e-06 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175254+0000", 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4609999999999996e-06 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175259+0000", 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999998e-06 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246277+0000", 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071018681 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246285+0000", 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.357e-06 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51003 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4083454120000001, 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071047076000000001, 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51003 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5499999999999998e-07 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175267+0000", 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1509999999999998e-06 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175274+0000", 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.426e-06 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175279+0000", 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2120000000000002e-06 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246306+0000", 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071026813999999994 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246312+0000", 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0179999999999998e-06 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.655 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51004 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175285+0000", 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4083250029999999, 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071052522000000007, 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51004 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.656 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175285+0000", 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175285+0000", 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175286+0000", 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.46e-07 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175288+0000", 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1299999999999999e-06 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175294+0000", 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.46e-06 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175299+0000", 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9749999999999998e-06 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246332+0000", 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071032895999999998 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246338+0000", 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7150000000000003e-06 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51005 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175305+0000", 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4083050740000003, 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071058287999999997, 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51005 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175305+0000", 2024-05-08T03:20:19.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175305+0000", 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175306+0000", 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175308+0000", 2024-05-08T03:20:19.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0470000000000001e-06 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175314+0000", 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3260000000000001e-06 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175319+0000", 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3159999999999996e-06 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246357+0000", 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071038089999999998 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246363+0000", 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0920000000000003e-06 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51006 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175326+0000", 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.408284074, 2024-05-08T03:20:19.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071064635000000001, 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51006 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175326+0000", 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175326+0000", 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175327+0000", 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9400000000000001e-07 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175329+0000", 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2809999999999998e-06 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175335+0000", 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4810000000000003e-06 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175340+0000", 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9169999999999997e-06 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246384+0000", 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071043998999999997 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246391+0000", 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5629999999999997e-06 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51008 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175346+0000", 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4082639090000004, 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071071253000000001, 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.667 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51008 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175346+0000", 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175346+0000", 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175347+0000", 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175349+0000", 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.041e-06 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175355+0000", 2024-05-08T03:20:19.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175360+0000", 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1919999999999996e-06 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246411+0000", 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071050786000000005 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246418+0000", 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6390000000000003e-06 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51012 3.8 3:1703486b:::rbd_header.1182fdc479eb:head [watch reconnect cookie 140222363529632 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.176104+0000", 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4075058880000002, 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074566035000000003, 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.671 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51012 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176104+0000", 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176104+0000", 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176105+0000", 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6300000000000005e-07 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176108+0000", 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3529999999999999e-06 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176116+0000", 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1529999999999997e-06 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176123+0000", 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9240000000000001e-06 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176170+0000", 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6412000000000002e-05 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176245+0000", 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4994999999999996e-05 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.250670+0000", 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074425535000000001 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51014 3.7 3.8f32ac47 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.176959+0000", 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.4066514940000001, 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069488783999999998, 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:19.675 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51014 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176959+0000", 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:19.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176959+0000", 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176960+0000", 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5700000000000001e-07 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176964+0000", 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1400000000000002e-06 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:19.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.177025+0000", 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1583999999999999e-05 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.177033+0000", 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1850000000000004e-06 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246440+0000", 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069406994 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:19.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246447+0000", 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.024e-06 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:19.679 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:19.679 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:19.681 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:19.682 INFO:teuthology.orchestra.run.smithi177.stdout:3155 write 0xcb27b64 thru 0xcb2a9bf (0x2e5c bytes) 2024-05-08T03:20:19.682 INFO:teuthology.orchestra.run.smithi177.stdout:3156 read 0x43656a thru 0x43dd39 (0x77d0 bytes) 2024-05-08T03:20:19.682 INFO:teuthology.orchestra.run.smithi177.stdout:3158 punch from 0x8d8eb5f to 0x8d9a7c0, (0xbc61 bytes) 2024-05-08T03:20:19.682 INFO:teuthology.orchestra.run.smithi177.stdout:3159 read 0x6846101 thru 0x685486c (0xe76c bytes) 2024-05-08T03:20:19.682 INFO:teuthology.orchestra.run.smithi177.stdout:3161 writesame 0x31781f0 thru 0x3185f1d data_size 0x3(0xdd2e bytes) 2024-05-08T03:20:19.682 INFO:teuthology.orchestra.run.smithi177.stdout:3162 write 0xe2956f3 thru 0xe29b5b6 (0x5ec4 bytes) 2024-05-08T03:20:19.683 INFO:teuthology.orchestra.run.smithi177.stdout:3163 trunc from 0xe29b5b7 to 0x696e1f9 2024-05-08T03:20:19.683 INFO:teuthology.orchestra.run.smithi177.stdout:3164 punch from 0x55fa88f to 0x5604332, (0x9aa3 bytes) 2024-05-08T03:20:19.683 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.683 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:19.678+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:19.683 INFO:teuthology.orchestra.run.smithi177.stdout:3166 writesame 0xd41f70a thru 0xd4285ae data_size 0x1(0x8ea5 bytes) 2024-05-08T03:20:19.684 INFO:teuthology.orchestra.run.smithi177.stdout:3167 read 0x675a400 thru 0x675df74 (0x3b75 bytes) 2024-05-08T03:20:19.688 INFO:teuthology.orchestra.run.smithi177.stdout:3169 write 0xd66587c thru 0xd667d05 (0x248a bytes) 2024-05-08T03:20:19.701 INFO:teuthology.orchestra.run.smithi177.stdout:3170 trunc from 0xd667d06 to 0x529dc24 2024-05-08T03:20:19.712 INFO:teuthology.orchestra.run.smithi177.stdout:3172 writesame 0x3f8b3a9 thru 0x3f8ebc1 data_size 0x1(0x3819 bytes) 2024-05-08T03:20:19.722 INFO:teuthology.orchestra.run.smithi177.stdout:3173 read 0x1b10176 thru 0x1b14df8 (0x4c83 bytes) 2024-05-08T03:20:19.722 INFO:teuthology.orchestra.run.smithi177.stdout:3176 writesame 0x1b24d3b thru 0x1b2daeb data_size 0x6b(0x8db1 bytes) 2024-05-08T03:20:19.732 INFO:teuthology.orchestra.run.smithi177.stdout:3177 write 0x76e67f7 thru 0x76eac16 (0x4420 bytes) 2024-05-08T03:20:19.743 INFO:teuthology.orchestra.run.smithi177.stdout:3178 trunc from 0x76eac17 to 0xe2d66a4 2024-05-08T03:20:19.749 INFO:teuthology.orchestra.run.smithi177.stdout:3179 read 0x9140998 thru 0x9141b85 (0x11ee bytes) 2024-05-08T03:20:19.749 INFO:teuthology.orchestra.run.smithi177.stdout:3181 write 0x73e9de thru 0x74382f (0x4e52 bytes) 2024-05-08T03:20:19.751 INFO:teuthology.orchestra.run.smithi177.stdout:3183 read 0xcebf08e thru 0xcecce8e (0xde01 bytes) 2024-05-08T03:20:19.751 INFO:teuthology.orchestra.run.smithi177.stdout:3184 read 0x888c399 thru 0x8897b3f (0xb7a7 bytes) 2024-05-08T03:20:19.751 INFO:teuthology.orchestra.run.smithi177.stdout:3185 read 0x3aa0c44 thru 0x3aae43a (0xd7f7 bytes) 2024-05-08T03:20:19.751 INFO:teuthology.orchestra.run.smithi177.stdout:3186 write 0xe2bc9c0 thru 0xe2ca602 (0xdc43 bytes) 2024-05-08T03:20:19.754 INFO:teuthology.orchestra.run.smithi177.stdout:3187 writesame 0x7db877f thru 0x7dc4cac data_size 0x5e(0xc52e bytes) 2024-05-08T03:20:19.769 INFO:teuthology.orchestra.run.smithi177.stdout:3189 trunc from 0xe2d66a4 to 0x4e26d0d 2024-05-08T03:20:19.781 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:19.779+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:19.793 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:19.793 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:19.794 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:19.794 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:3191 write 0xcd0ac64 thru 0xcd19d80 (0xf11d bytes) 2024-05-08T03:20:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:3193 read 0x1f12f76 thru 0x1f162ac (0x3337 bytes) 2024-05-08T03:20:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:3194 writesame 0x71e17f8 thru 0x71efa11 data_size 0x21(0xe21a bytes) 2024-05-08T03:20:19.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:3195 write 0x8583c09 thru 0x8593b93 (0xff8b bytes) 2024-05-08T03:20:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:3196 punch from 0x2bc7923 to 0x2bcd8b7, (0x5f94 bytes) 2024-05-08T03:20:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:3198 read 0x74173ea thru 0x741be22 (0x4a39 bytes) 2024-05-08T03:20:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:3200 punch from 0xb67491 to 0xb6f37a, (0x7ee9 bytes) 2024-05-08T03:20:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:3202 punch from 0x5fe35af to 0x5fea902, (0x7353 bytes) 2024-05-08T03:20:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:3203 read 0x9052e81 thru 0x90544ae (0x162e bytes) 2024-05-08T03:20:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:3205 read 0x4b70356 thru 0x4b70c9f (0x94a bytes) 2024-05-08T03:20:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:3206 read 0x5a063af thru 0x5a0e497 (0x80e9 bytes) 2024-05-08T03:20:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:3209 writesame 0x63a6603 thru 0x63aebac data_size 0x9(0x85aa bytes) 2024-05-08T03:20:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:3210 writesame 0x6c3a024 thru 0x6c3b566 data_size 0x1(0x1543 bytes) 2024-05-08T03:20:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:3211 punch from 0x201c4ea to 0x2022130, (0x5c46 bytes) 2024-05-08T03:20:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:3212 trunc from 0xcd19d81 to 0xbea8eca 2024-05-08T03:20:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:19.831 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:3214 read 0x36e7955 thru 0x36ee3ee (0x6a9a bytes) 2024-05-08T03:20:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:3215 read 0xb2b48a thru 0xb3810c (0xcc83 bytes) 2024-05-08T03:20:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:3216 read 0xae0f9f thru 0xae7265 (0x62c7 bytes) 2024-05-08T03:20:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:3218 trunc from 0xbea8eca to 0x4cf1fae 2024-05-08T03:20:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:19.856 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:3219 read 0x4485ae4 thru 0x4494126 (0xe643 bytes) 2024-05-08T03:20:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:3220 writesame 0x2e12876 thru 0x2e13ae6 data_size 0x1(0x1271 bytes) 2024-05-08T03:20:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:19.875 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:3221 write 0x59294f0 thru 0x592fab0 (0x65c1 bytes) 2024-05-08T03:20:19.887 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:19.885+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:19.894 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:3222 write 0xc88a461 thru 0xc88f8df (0x547f bytes) 2024-05-08T03:20:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:3223 read 0xac99456 thru 0xaca5864 (0xc40f bytes) 2024-05-08T03:20:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:19.915 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:19.915 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:3224 trunc from 0xc88f8e0 to 0x531083e 2024-05-08T03:20:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:3225 read 0x365c494 thru 0x3665182 (0x8cef bytes) 2024-05-08T03:20:19.941 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:19.941 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:19.944 INFO:teuthology.orchestra.run.smithi177.stdout:3226 write 0xb5b277b thru 0xb5beb3e (0xc3c4 bytes) 2024-05-08T03:20:19.952 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:19.952 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:19.956 INFO:teuthology.orchestra.run.smithi177.stdout:3227 write 0xd195425 thru 0xd1a52b6 (0xfe92 bytes) 2024-05-08T03:20:19.961 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:19.962 INFO:teuthology.orchestra.run.smithi177.stdout:3230 write 0x24b4747 thru 0x24bf3ef (0xaca9 bytes) 2024-05-08T03:20:19.963 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:19.963 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:19.965 INFO:teuthology.orchestra.run.smithi177.stdout:3232 punch from 0x89ef83c to 0x89f357e, (0x3d42 bytes) 2024-05-08T03:20:19.965 INFO:teuthology.orchestra.run.smithi177.stdout:3233 write 0x8f4c940 thru 0x8f566ff (0x9dc0 bytes) 2024-05-08T03:20:19.966 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:19.967 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:19.970 INFO:teuthology.orchestra.run.smithi177.stdout:3234 writesame 0x43e789e thru 0x43e8c40 data_size 0x1(0x13a3 bytes) 2024-05-08T03:20:19.976 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.976 INFO:teuthology.orchestra.run.smithi177.stdout:3238 read 0x53757df thru 0x538572f (0xff51 bytes) 2024-05-08T03:20:19.976 INFO:teuthology.orchestra.run.smithi177.stdout:3240 write 0x20dcc7e thru 0x20e2e10 (0x6193 bytes) 2024-05-08T03:20:19.977 INFO:teuthology.orchestra.run.smithi177.stdout:3241 write 0x24a27c7 thru 0x24a5acc (0x3306 bytes) 2024-05-08T03:20:19.979 INFO:teuthology.orchestra.run.smithi177.stdout:3242 read 0x4440aa8 thru 0x4447bbe (0x7117 bytes) 2024-05-08T03:20:19.979 INFO:teuthology.orchestra.run.smithi177.stdout:3244 read 0x8dabf86 thru 0x8db71e3 (0xb25e bytes) 2024-05-08T03:20:19.979 INFO:teuthology.orchestra.run.smithi177.stdout:3245 read 0x716c23e thru 0x7173a1d (0x77e0 bytes) 2024-05-08T03:20:19.979 INFO:teuthology.orchestra.run.smithi177.stdout:3246 read 0x89e6c8f thru 0x89e9d0e (0x3080 bytes) 2024-05-08T03:20:19.979 INFO:teuthology.orchestra.run.smithi177.stdout:3247 writesame 0x66a263f thru 0x66a8a43 data_size 0x5(0x6405 bytes) 2024-05-08T03:20:19.983 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:19.981+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:19.985 INFO:teuthology.orchestra.run.smithi177.stdout:3249 read 0x23aa224 thru 0x23b0a11 (0x67ee bytes) 2024-05-08T03:20:19.985 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.985 INFO:teuthology.orchestra.run.smithi177.stdout:3251 write 0xbce251 thru 0xbd3713 (0x54c3 bytes) 2024-05-08T03:20:19.988 INFO:teuthology.orchestra.run.smithi177.stdout:3253 punch from 0x6c792fe to 0x6c797e5, (0x4e7 bytes) 2024-05-08T03:20:19.988 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:19.988 INFO:teuthology.orchestra.run.smithi177.stdout:3258 write 0x53c43e3 thru 0x53c6c92 (0x28b0 bytes) 2024-05-08T03:20:19.993 INFO:teuthology.orchestra.run.smithi177.stdout:3259 read 0xa8d2c92 thru 0xa8d9794 (0x6b03 bytes) 2024-05-08T03:20:19.993 INFO:teuthology.orchestra.run.smithi177.stdout:3261 write 0x47ff345 thru 0x480a866 (0xb522 bytes) 2024-05-08T03:20:20.011 INFO:teuthology.orchestra.run.smithi177.stdout:3262 trunc from 0xd1a52b7 to 0x59db753 2024-05-08T03:20:20.022 INFO:teuthology.orchestra.run.smithi177.stdout:3263 punch from 0xfe3000 to 0xff2c84, (0xfc84 bytes) 2024-05-08T03:20:20.031 INFO:teuthology.orchestra.run.smithi177.stdout:3264 trunc from 0x59db753 to 0xd408676 2024-05-08T03:20:20.037 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.039 INFO:teuthology.orchestra.run.smithi177.stdout:3267 trunc from 0xd408676 to 0xb842ca2 2024-05-08T03:20:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:3268 write 0x932e536 thru 0x93383d9 (0x9ea4 bytes) 2024-05-08T03:20:20.050 INFO:teuthology.orchestra.run.smithi177.stdout:3269 trunc from 0xb842ca2 to 0xe070d1a 2024-05-08T03:20:20.056 INFO:teuthology.orchestra.run.smithi177.stdout:3271 write 0x422c5a8 thru 0x423a8d6 (0xe32f bytes) 2024-05-08T03:20:20.058 INFO:teuthology.orchestra.run.smithi177.stdout:3272 punch from 0xa7b814d to 0xa7bd6dd, (0x5590 bytes) 2024-05-08T03:20:20.069 INFO:teuthology.orchestra.run.smithi177.stdout:3273 trunc from 0xe070d1a to 0x4479ce1 2024-05-08T03:20:20.083 INFO:teuthology.orchestra.run.smithi177.stdout:3274 write 0x1be898d thru 0x1bf59c7 (0xd03b bytes) 2024-05-08T03:20:20.083 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:20.081+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:20.098 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:20:20.100 INFO:teuthology.orchestra.run.smithi177.stdout:3276 writesame 0xd0e7043 thru 0xd0e8629 data_size 0x15(0x15e7 bytes) 2024-05-08T03:20:20.100 INFO:teuthology.orchestra.run.smithi177.stdout:3277 write 0x8e982dd thru 0x8e9864c (0x370 bytes) 2024-05-08T03:20:20.107 INFO:teuthology.orchestra.run.smithi177.stdout:3278 writesame 0xc9d9f21 thru 0xc9dbb95 data_size 0x1f(0x1c75 bytes) 2024-05-08T03:20:20.110 INFO:teuthology.orchestra.run.smithi177.stdout:3279 write 0xd049fb9 thru 0xd05632c (0xc374 bytes) 2024-05-08T03:20:20.111 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.111 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.113 INFO:teuthology.orchestra.run.smithi177.stdout:3280 read 0xbab7927 thru 0xbac13c8 (0x9aa2 bytes) 2024-05-08T03:20:20.114 INFO:teuthology.orchestra.run.smithi177.stdout:3281 trunc from 0xd0e862a to 0xc2da5d1 2024-05-08T03:20:20.120 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:20.125 INFO:teuthology.orchestra.run.smithi177.stdout:3282 punch from 0x79bac33 to 0x79bd7d2, (0x2b9f bytes) 2024-05-08T03:20:20.125 INFO:teuthology.orchestra.run.smithi177.stdout:3283 read 0x32ca941 thru 0x32d72b0 (0xc970 bytes) 2024-05-08T03:20:20.125 INFO:teuthology.orchestra.run.smithi177.stdout:3284 read 0xb652dec thru 0xb65ac29 (0x7e3e bytes) 2024-05-08T03:20:20.125 INFO:teuthology.orchestra.run.smithi177.stdout:3285 writesame 0xb398279 thru 0xb3a47c9 data_size 0x1(0xc551 bytes) 2024-05-08T03:20:20.153 INFO:teuthology.orchestra.run.smithi177.stdout:3287 read 0x68bbff8 thru 0x68bf116 (0x311f bytes) 2024-05-08T03:20:20.153 INFO:teuthology.orchestra.run.smithi177.stdout:3290 read 0x6a9a144 thru 0x6a9df30 (0x3ded bytes) 2024-05-08T03:20:20.153 INFO:teuthology.orchestra.run.smithi177.stdout:3291 writesame 0xb40b4e3 thru 0xb417f86 data_size 0x84(0xcaa4 bytes) 2024-05-08T03:20:20.156 INFO:teuthology.orchestra.run.smithi177.stdout:3294 write 0x82fd6e0 thru 0x8309ead (0xc7ce bytes) 2024-05-08T03:20:20.159 INFO:teuthology.orchestra.run.smithi177.stdout:3295 trunc from 0xc2da5d1 to 0xd965c10 2024-05-08T03:20:20.164 INFO:teuthology.orchestra.run.smithi177.stdout:3296 write 0x2a3beb1 thru 0x2a3feda (0x402a bytes) 2024-05-08T03:20:20.166 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.166 INFO:teuthology.orchestra.run.smithi177.stdout:3298 write 0x4c7a288 thru 0x4c89e3b (0xfbb4 bytes) 2024-05-08T03:20:20.168 INFO:teuthology.orchestra.run.smithi177.stdout:3299 write 0x27c1a3f thru 0x27cbd6d (0xa32f bytes) 2024-05-08T03:20:20.169 INFO:teuthology.orchestra.run.smithi177.stdout:3300 write 0x76e6da5 thru 0x76f1474 (0xa6d0 bytes) 2024-05-08T03:20:20.171 INFO:teuthology.orchestra.run.smithi177.stdout:3301 write 0x84fe9dc thru 0x850a54a (0xbb6f bytes) 2024-05-08T03:20:20.174 INFO:teuthology.orchestra.run.smithi177.stdout:3302 write 0x35cc54a thru 0x35cc6ce (0x185 bytes) 2024-05-08T03:20:20.176 INFO:teuthology.orchestra.run.smithi177.stdout:3303 read 0x3aee340 thru 0x3af9088 (0xad49 bytes) 2024-05-08T03:20:20.182 INFO:teuthology.orchestra.run.smithi177.stdout:3304 trunc from 0xd965c10 to 0x92e1ef9 2024-05-08T03:20:20.184 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:20.182+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:20.200 INFO:teuthology.orchestra.run.smithi177.stdout:3305 write 0xcb8a796 thru 0xcb97b7b (0xd3e6 bytes) 2024-05-08T03:20:20.204 INFO:teuthology.orchestra.run.smithi177.stdout:3306 write 0x7f51965 thru 0x7f55c85 (0x4321 bytes) 2024-05-08T03:20:20.207 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.207 INFO:teuthology.orchestra.run.smithi177.stdout:3310 read 0x17b84cf thru 0x17c83c5 (0xfef7 bytes) 2024-05-08T03:20:20.208 INFO:teuthology.orchestra.run.smithi177.stdout:3311 read 0x306237d thru 0x3068010 (0x5c94 bytes) 2024-05-08T03:20:20.214 INFO:teuthology.orchestra.run.smithi177.stdout:3313 writesame 0xd41dec1 thru 0xd424866 data_size 0x1(0x69a6 bytes) 2024-05-08T03:20:20.214 INFO:teuthology.orchestra.run.smithi177.stdout:3314 read 0xd394110 thru 0xd3958ca (0x17bb bytes) 2024-05-08T03:20:20.214 INFO:teuthology.orchestra.run.smithi177.stdout:3316 read 0x5a2777e thru 0x5a28678 (0xefb bytes) 2024-05-08T03:20:20.214 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.214 INFO:teuthology.orchestra.run.smithi177.stdout:3318 write 0xb55d398 thru 0xb56b313 (0xdf7c bytes) 2024-05-08T03:20:20.215 INFO:teuthology.orchestra.run.smithi177.stdout:3319 read 0x6b895fd thru 0x6b8f842 (0x6246 bytes) 2024-05-08T03:20:20.216 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.216 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.216 INFO:teuthology.orchestra.run.smithi177.stdout:3325 writesame 0x20c4ac2 thru 0x20cdf9f data_size 0x4a(0x94de bytes) 2024-05-08T03:20:20.218 INFO:teuthology.orchestra.run.smithi177.stdout:3328 read 0x693d26 thru 0x6a00e2 (0xc3bd bytes) 2024-05-08T03:20:20.218 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.219 INFO:teuthology.orchestra.run.smithi177.stdout:3331 read 0xc2b86f2 thru 0xc2c57a0 (0xd0af bytes) 2024-05-08T03:20:20.219 INFO:teuthology.orchestra.run.smithi177.stdout:3332 write 0x81e131b thru 0x81e3ebe (0x2ba4 bytes) 2024-05-08T03:20:20.224 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.224 INFO:teuthology.orchestra.run.smithi177.stdout:3334 punch from 0x35b53a3 to 0x35bfa34, (0xa691 bytes) 2024-05-08T03:20:20.224 INFO:teuthology.orchestra.run.smithi177.stdout:3335 read 0xb68a427 thru 0xb68bb04 (0x16de bytes) 2024-05-08T03:20:20.231 INFO:teuthology.orchestra.run.smithi177.stdout:3336 writesame 0xe834316 thru 0xe83d3d6 data_size 0x1(0x90c1 bytes) 2024-05-08T03:20:20.232 INFO:teuthology.orchestra.run.smithi177.stdout:3337 punch from 0x91eff66 to 0x91f06af, (0x749 bytes) 2024-05-08T03:20:20.232 INFO:teuthology.orchestra.run.smithi177.stdout:3338 write 0x57d10a3 thru 0x57d204c (0xfaa bytes) 2024-05-08T03:20:20.233 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.233 INFO:teuthology.orchestra.run.smithi177.stdout:3340 writesame 0xe4d0fc7 thru 0xe4d5bf9 data_size 0x1(0x4c33 bytes) 2024-05-08T03:20:20.245 INFO:teuthology.orchestra.run.smithi177.stdout:3341 write 0x7e9101e thru 0x7e910da (0xbd bytes) 2024-05-08T03:20:20.247 INFO:teuthology.orchestra.run.smithi177.stdout:3343 write 0x14f6f9d thru 0x1506239 (0xf29d bytes) 2024-05-08T03:20:20.249 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.249 INFO:teuthology.orchestra.run.smithi177.stdout:3346 read 0xd9012f8 thru 0xd90da5b (0xc764 bytes) 2024-05-08T03:20:20.249 INFO:teuthology.orchestra.run.smithi177.stdout:3347 punch from 0xe1ed6cf to 0xe1edb4c, (0x47d bytes) 2024-05-08T03:20:20.250 INFO:teuthology.orchestra.run.smithi177.stdout:3349 write 0xd330843 thru 0xd335dca (0x5588 bytes) 2024-05-08T03:20:20.254 INFO:teuthology.orchestra.run.smithi177.stdout:3350 read 0x8eba946 thru 0x8ebe22b (0x38e6 bytes) 2024-05-08T03:20:20.255 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.267 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.267 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.271 INFO:teuthology.orchestra.run.smithi177.stdout:3355 trunc from 0xe83d3d7 to 0x8870c1 2024-05-08T03:20:20.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:20.284 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:20.283+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:20.315 INFO:teuthology.orchestra.run.smithi177.stdout:3356 read 0x2bfb4e thru 0x2c151a (0x19cd bytes) 2024-05-08T03:20:20.339 INFO:teuthology.orchestra.run.smithi177.stdout:3359 writesame 0xe02c1ab thru 0xe0349e3 data_size 0x1(0x8839 bytes) 2024-05-08T03:20:20.353 INFO:teuthology.orchestra.run.smithi177.stdout:3360 trunc from 0xe0349e4 to 0x2551271 2024-05-08T03:20:20.365 INFO:teuthology.orchestra.run.smithi177.stdout:3361 read 0x24ff843 thru 0x250265d (0x2e1b bytes) 2024-05-08T03:20:20.366 INFO:teuthology.orchestra.run.smithi177.stdout:3364 read 0x237322f thru 0x2380f47 (0xdd19 bytes) 2024-05-08T03:20:20.377 INFO:teuthology.orchestra.run.smithi177.stdout:3366 write 0x734f887 thru 0x735624f (0x69c9 bytes) 2024-05-08T03:20:20.380 INFO:teuthology.orchestra.run.smithi177.stdout:3369 read 0x2c7208b thru 0x2c78071 (0x5fe7 bytes) 2024-05-08T03:20:20.380 INFO:teuthology.orchestra.run.smithi177.stdout:3370 read 0x5ef7e6 thru 0x5f451b (0x4d36 bytes) 2024-05-08T03:20:20.381 INFO:teuthology.orchestra.run.smithi177.stdout:3372 read 0x843597 thru 0x850723 (0xd18d bytes) 2024-05-08T03:20:20.383 INFO:teuthology.orchestra.run.smithi177.stdout:3375 trunc from 0x7356250 to 0x5dbd99f 2024-05-08T03:20:20.386 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:20.384+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:20.386 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.386 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.392 INFO:teuthology.orchestra.run.smithi177.stdout:3376 read 0x3372019 thru 0x337d88d (0xb875 bytes) 2024-05-08T03:20:20.392 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:20.410 INFO:teuthology.orchestra.run.smithi177.stdout:3378 write 0xe7a7ee1 thru 0xe7ad6fc (0x581c bytes) 2024-05-08T03:20:20.417 INFO:teuthology.orchestra.run.smithi177.stdout:3380 punch from 0xb4c638f to 0xb4cb83a, (0x54ab bytes) 2024-05-08T03:20:20.417 INFO:teuthology.orchestra.run.smithi177.stdout:3381 read 0xe925c thru 0xf7011 (0xddb6 bytes) 2024-05-08T03:20:20.432 INFO:teuthology.orchestra.run.smithi177.stdout:3383 trunc from 0xe7ad6fd to 0xafc9b4 2024-05-08T03:20:20.458 INFO:teuthology.orchestra.run.smithi177.stdout:3384 punch from 0x765c89 to 0x766b55, (0xecc bytes) 2024-05-08T03:20:20.465 INFO:teuthology.orchestra.run.smithi177.stdout:3385 trunc from 0xafc9b4 to 0x691a9f3 2024-05-08T03:20:20.471 INFO:teuthology.orchestra.run.smithi177.stdout:3386 punch from 0x26cbd53 to 0x26d2e84, (0x7131 bytes) 2024-05-08T03:20:20.477 INFO:teuthology.orchestra.run.smithi177.stdout:3387 trunc from 0x691a9f3 to 0xbedc4ab 2024-05-08T03:20:20.485 INFO:teuthology.orchestra.run.smithi177.stdout:3388 read 0x224b957 thru 0x2257518 (0xbbc2 bytes) 2024-05-08T03:20:20.485 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.485 INFO:teuthology.orchestra.run.smithi177.stdout:3390 write 0x55ea827 thru 0x55f617d (0xb957 bytes) 2024-05-08T03:20:20.486 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:20.484+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:20.491 INFO:teuthology.orchestra.run.smithi177.stdout:3392 punch from 0xb687783 to 0xb688ac3, (0x1340 bytes) 2024-05-08T03:20:20.496 INFO:teuthology.orchestra.run.smithi177.stdout:3393 trunc from 0xbedc4ab to 0x7c84a27 2024-05-08T03:20:20.513 INFO:teuthology.orchestra.run.smithi177.stdout:3394 writesame 0xcb59a58 thru 0xcb5dbb4 data_size 0x1(0x415d bytes) 2024-05-08T03:20:20.520 INFO:teuthology.orchestra.run.smithi177.stdout:3395 write 0xe53e589 thru 0xe54adb2 (0xc82a bytes) 2024-05-08T03:20:20.524 INFO:teuthology.orchestra.run.smithi177.stdout:3396 read 0x2f046c4 thru 0x2f10d1e (0xc65b bytes) 2024-05-08T03:20:20.525 INFO:teuthology.orchestra.run.smithi177.stdout:3397 trunc from 0xe54adb3 to 0xd06bb0e 2024-05-08T03:20:20.531 INFO:teuthology.orchestra.run.smithi177.stdout:3398 read 0x4eb5494 thru 0x4eb7989 (0x24f6 bytes) 2024-05-08T03:20:20.541 INFO:teuthology.orchestra.run.smithi177.stdout:3399 trunc from 0xd06bb0e to 0x4c65355 2024-05-08T03:20:20.548 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.549 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.551 INFO:teuthology.orchestra.run.smithi177.stdout:3401 read 0x401be13 thru 0x401cde2 (0xfd0 bytes) 2024-05-08T03:20:20.551 INFO:teuthology.orchestra.run.smithi177.stdout:3402 writesame 0x3595cbc thru 0x35a412d data_size 0x51(0xe472 bytes) 2024-05-08T03:20:20.555 INFO:teuthology.orchestra.run.smithi177.stdout:3403 trunc from 0x4c65355 to 0x544d7c9 2024-05-08T03:20:20.560 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:20.573 INFO:teuthology.orchestra.run.smithi177.stdout:3404 write 0x9c35a02 thru 0x9c3a1cf (0x47ce bytes) 2024-05-08T03:20:20.576 INFO:teuthology.orchestra.run.smithi177.stdout:3405 punch from 0x94c5fd8 to 0x94d4324, (0xe34c bytes) 2024-05-08T03:20:20.576 INFO:teuthology.orchestra.run.smithi177.stdout:3406 read 0x7979e76 thru 0x7989265 (0xf3f0 bytes) 2024-05-08T03:20:20.584 INFO:teuthology.orchestra.run.smithi177.stdout:3407 writesame 0xba18963 thru 0xba278b0 data_size 0x1(0xef4e bytes) 2024-05-08T03:20:20.584 INFO:teuthology.orchestra.run.smithi177.stdout:3408 write 0xa550ed9 thru 0xa55b856 (0xa97e bytes) 2024-05-08T03:20:20.587 INFO:teuthology.orchestra.run.smithi177.stdout:3411 write 0x91e31eb thru 0x91e458e (0x13a4 bytes) 2024-05-08T03:20:20.589 INFO:teuthology.orchestra.run.smithi177.stdout:3412 read 0x7e459da thru 0x7e4cf39 (0x7560 bytes) 2024-05-08T03:20:20.594 INFO:teuthology.orchestra.run.smithi177.stdout:3415 trunc from 0xba278b1 to 0x6edbf95 2024-05-08T03:20:20.601 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:20.599+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:20.615 INFO:teuthology.orchestra.run.smithi177.stdout:3416 trunc from 0x6edbf95 to 0x1df23c1 2024-05-08T03:20:20.639 INFO:teuthology.orchestra.run.smithi177.stdout:3417 write 0x48b9689 thru 0x48c114d (0x7ac5 bytes) 2024-05-08T03:20:20.642 INFO:teuthology.orchestra.run.smithi177.stdout:3419 writesame 0x3dd7aaf thru 0x3ddc580 data_size 0x2(0x4ad2 bytes) 2024-05-08T03:20:20.648 INFO:teuthology.orchestra.run.smithi177.stdout:3420 read 0xe17b7e thru 0xe2564a (0xdacd bytes) 2024-05-08T03:20:20.648 INFO:teuthology.orchestra.run.smithi177.stdout:3422 punch from 0x15d18dc to 0x15e10a7, (0xf7cb bytes) 2024-05-08T03:20:20.648 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.658 INFO:teuthology.orchestra.run.smithi177.stdout:3424 trunc from 0x48c114e to 0xd1567c9 2024-05-08T03:20:20.663 INFO:teuthology.orchestra.run.smithi177.stdout:3425 write 0x556ec96 thru 0x5571cf4 (0x305f bytes) 2024-05-08T03:20:20.665 INFO:teuthology.orchestra.run.smithi177.stdout:3426 read 0x4551322 thru 0x4556e13 (0x5af2 bytes) 2024-05-08T03:20:20.665 INFO:teuthology.orchestra.run.smithi177.stdout:3427 read 0x169eb5c thru 0x16a2b18 (0x3fbd bytes) 2024-05-08T03:20:20.665 INFO:teuthology.orchestra.run.smithi177.stdout:3428 read 0xc436bfb thru 0xc441e7c (0xb282 bytes) 2024-05-08T03:20:20.665 INFO:teuthology.orchestra.run.smithi177.stdout:3430 read 0x1ce5034 thru 0x1ce967f (0x464c bytes) 2024-05-08T03:20:20.668 INFO:teuthology.orchestra.run.smithi177.stdout:3431 trunc from 0xd1567c9 to 0xaffcb18 2024-05-08T03:20:20.673 INFO:teuthology.orchestra.run.smithi177.stdout:3433 read 0x86b1605 thru 0x86bdf33 (0xc92f bytes) 2024-05-08T03:20:20.674 INFO:teuthology.orchestra.run.smithi177.stdout:3434 flatten 2024-05-08T03:20:20.680 INFO:teuthology.orchestra.run.smithi177.stdout:3436 punch from 0x4f4a0bc to 0x4f4bfa7, (0x1eeb bytes) 2024-05-08T03:20:20.680 INFO:teuthology.orchestra.run.smithi177.stdout:3437 read 0x38c3a3f thru 0x38ce8f0 (0xaeb2 bytes) 2024-05-08T03:20:20.680 INFO:teuthology.orchestra.run.smithi177.stdout:3438 read 0x77761e6 thru 0x778176f (0xb58a bytes) 2024-05-08T03:20:20.681 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.681 INFO:teuthology.orchestra.run.smithi177.stdout:3442 read 0xaeaa677 thru 0xaeba42f (0xfdb9 bytes) 2024-05-08T03:20:20.681 INFO:teuthology.orchestra.run.smithi177.stdout:3443 read 0xe98be thru 0xf5065 (0xb7a8 bytes) 2024-05-08T03:20:20.681 INFO:teuthology.orchestra.run.smithi177.stdout:3444 read 0xa4c56e5 thru 0xa4d11be (0xbada bytes) 2024-05-08T03:20:20.681 INFO:teuthology.orchestra.run.smithi177.stdout:3446 write 0x34bdfe3 thru 0x34bf200 (0x121e bytes) 2024-05-08T03:20:20.684 INFO:teuthology.orchestra.run.smithi177.stdout:3447 write 0x8d4d294 thru 0x8d57254 (0x9fc1 bytes) 2024-05-08T03:20:20.687 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:20.686+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:20.689 INFO:teuthology.orchestra.run.smithi177.stdout:3448 writesame 0x1ff6f55 thru 0x2002a76 data_size 0x11(0xbb22 bytes) 2024-05-08T03:20:20.704 INFO:teuthology.orchestra.run.smithi177.stdout:3449 trunc from 0xaffcb18 to 0x29fd43a 2024-05-08T03:20:20.711 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.712 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.720 INFO:teuthology.orchestra.run.smithi177.stdout:3450 trunc from 0x29fd43a to 0x508ff02 2024-05-08T03:20:20.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:20.725 INFO:teuthology.orchestra.run.smithi177.stdout:3451 read 0x2a268bf thru 0x2a303c2 (0x9b04 bytes) 2024-05-08T03:20:20.725 INFO:teuthology.orchestra.run.smithi177.stdout:3454 read 0x3a9b878 thru 0x3aa33b3 (0x7b3c bytes) 2024-05-08T03:20:20.731 INFO:teuthology.orchestra.run.smithi177.stdout:3456 write 0x5d30262 thru 0x5d32899 (0x2638 bytes) 2024-05-08T03:20:20.735 INFO:teuthology.orchestra.run.smithi177.stdout:3457 punch from 0x1b4ce83 to 0x1b4e7f1, (0x196e bytes) 2024-05-08T03:20:20.746 INFO:teuthology.orchestra.run.smithi177.stdout:3458 writesame 0x915719b thru 0x915d2e4 data_size 0xe(0x614a bytes) 2024-05-08T03:20:20.746 INFO:teuthology.orchestra.run.smithi177.stdout:3459 writesame 0x4a91cc3 thru 0x4a9fed6 data_size 0x6a(0xe214 bytes) 2024-05-08T03:20:20.750 INFO:teuthology.orchestra.run.smithi177.stdout:3460 read 0x6fb5a55 thru 0x6fc3826 (0xddd2 bytes) 2024-05-08T03:20:20.750 INFO:teuthology.orchestra.run.smithi177.stdout:3461 read 0x8a93c72 thru 0x8a96a53 (0x2de2 bytes) 2024-05-08T03:20:20.750 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.750 INFO:teuthology.orchestra.run.smithi177.stdout:3464 read 0x3dd5360 thru 0x3ddb471 (0x6112 bytes) 2024-05-08T03:20:20.750 INFO:teuthology.orchestra.run.smithi177.stdout:3465 read 0x4c36701 thru 0x4c40502 (0x9e02 bytes) 2024-05-08T03:20:20.750 INFO:teuthology.orchestra.run.smithi177.stdout:3466 read 0x7b862c6 thru 0x7b92e76 (0xcbb1 bytes) 2024-05-08T03:20:20.751 INFO:teuthology.orchestra.run.smithi177.stdout:3468 read 0x40ecf10 thru 0x40f2ae5 (0x5bd6 bytes) 2024-05-08T03:20:20.751 INFO:teuthology.orchestra.run.smithi177.stdout:3469 writesame 0x3c0a5db thru 0x3c12b79 data_size 0x1(0x859f bytes) 2024-05-08T03:20:20.765 INFO:teuthology.orchestra.run.smithi177.stdout:3470 writesame 0x8c0d109 thru 0x8c11ecd data_size 0x1(0x4dc5 bytes) 2024-05-08T03:20:20.776 INFO:teuthology.orchestra.run.smithi177.stdout:3471 punch from 0x4585a1c to 0x4589692, (0x3c76 bytes) 2024-05-08T03:20:20.776 INFO:teuthology.orchestra.run.smithi177.stdout:3474 write 0x5325c2f thru 0x5330fee (0xb3c0 bytes) 2024-05-08T03:20:20.788 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:20.786+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:20.788 INFO:teuthology.orchestra.run.smithi177.stdout:3475 writesame 0xc8a5896 thru 0xc8a928d data_size 0x46(0x39f8 bytes) 2024-05-08T03:20:20.798 INFO:teuthology.orchestra.run.smithi177.stdout:3476 trunc from 0xc8a928e to 0x4a4c772 2024-05-08T03:20:20.814 INFO:teuthology.orchestra.run.smithi177.stdout:3477 trunc from 0x4a4c772 to 0x6dcb12f 2024-05-08T03:20:20.833 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.833 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.836 INFO:teuthology.orchestra.run.smithi177.stdout:3478 write 0xe25de19 thru 0xe26598a (0x7b72 bytes) 2024-05-08T03:20:20.841 INFO:teuthology.orchestra.run.smithi177.stdout:3479 read 0x92c3ca thru 0x936b1a (0xa751 bytes) 2024-05-08T03:20:20.842 INFO:teuthology.orchestra.run.smithi177.stdout:3480 write 0xbb141ef thru 0xbb19a2f (0x5841 bytes) 2024-05-08T03:20:20.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:20.845 INFO:teuthology.orchestra.run.smithi177.stdout:3481 read 0xd95ff6a thru 0xd96b858 (0xb8ef bytes) 2024-05-08T03:20:20.845 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.845 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.845 INFO:teuthology.orchestra.run.smithi177.stdout:3484 read 0xd511817 thru 0xd5131de (0x19c8 bytes) 2024-05-08T03:20:20.845 INFO:teuthology.orchestra.run.smithi177.stdout:3485 writesame 0x5566150 thru 0x556e0f6 data_size 0x3(0x7fa7 bytes) 2024-05-08T03:20:20.851 INFO:teuthology.orchestra.run.smithi177.stdout:3486 punch from 0x8bdeed1 to 0x8be88da, (0x9a09 bytes) 2024-05-08T03:20:20.851 INFO:teuthology.orchestra.run.smithi177.stdout:3488 read 0xdbe210e thru 0xdbf0cd5 (0xebc8 bytes) 2024-05-08T03:20:20.851 INFO:teuthology.orchestra.run.smithi177.stdout:3489 read 0x1a8527a thru 0x1a8868f (0x3416 bytes) 2024-05-08T03:20:20.851 INFO:teuthology.orchestra.run.smithi177.stdout:3490 read 0xa334bf3 thru 0xa341416 (0xc824 bytes) 2024-05-08T03:20:20.852 INFO:teuthology.orchestra.run.smithi177.stdout:3491 punch from 0xd689643 to 0xd69112a, (0x7ae7 bytes) 2024-05-08T03:20:20.852 INFO:teuthology.orchestra.run.smithi177.stdout:3492 read 0xd69fe07 thru 0xd6ac934 (0xcb2e bytes) 2024-05-08T03:20:20.852 INFO:teuthology.orchestra.run.smithi177.stdout:3493 read 0x7f35bb5 thru 0x7f3b16d (0x55b9 bytes) 2024-05-08T03:20:20.852 INFO:teuthology.orchestra.run.smithi177.stdout:3494 write 0x1815d06 thru 0x181ae2d (0x5128 bytes) 2024-05-08T03:20:20.854 INFO:teuthology.orchestra.run.smithi177.stdout:3496 punch from 0xd0abfb9 to 0xd0bbc18, (0xfc5f bytes) 2024-05-08T03:20:20.854 INFO:teuthology.orchestra.run.smithi177.stdout:3497 writesame 0x6725910 thru 0x67302c8 data_size 0x7(0xa9b9 bytes) 2024-05-08T03:20:20.863 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.863 INFO:teuthology.orchestra.run.smithi177.stdout:3499 read 0x968f140 thru 0x96966db (0x759c bytes) 2024-05-08T03:20:20.864 INFO:teuthology.orchestra.run.smithi177.stdout:3503 read 0x77751f2 thru 0x777dc33 (0x8a42 bytes) 2024-05-08T03:20:20.864 INFO:teuthology.orchestra.run.smithi177.stdout:3504 punch from 0xa26aeac to 0xa2798e4, (0xea38 bytes) 2024-05-08T03:20:20.864 INFO:teuthology.orchestra.run.smithi177.stdout:3505 punch from 0x2c399a0 to 0x2c482c1, (0xe921 bytes) 2024-05-08T03:20:20.864 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.880 INFO:teuthology.orchestra.run.smithi177.stdout:3507 trunc from 0xe26598b to 0x41b7e3 2024-05-08T03:20:20.904 INFO:teuthology.orchestra.run.smithi177.stdout:3508 trunc from 0x41b7e3 to 0x59543c 2024-05-08T03:20:20.920 INFO:teuthology.orchestra.run.smithi177.stdout:3509 writesame 0x331e9aa thru 0x332bda8 data_size 0x1(0xd3ff bytes) 2024-05-08T03:20:20.926 INFO:teuthology.orchestra.run.smithi177.stdout:3511 writesame 0x3cdafdf thru 0x3ce1b5c data_size 0x1(0x6b7e bytes) 2024-05-08T03:20:20.926 INFO:teuthology.orchestra.run.smithi177.stdout:3514 writesame 0x133a89a thru 0x13420ba data_size 0x11(0x7821 bytes) 2024-05-08T03:20:20.927 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:20.925+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:20.935 INFO:teuthology.orchestra.run.smithi177.stdout:3516 trunc from 0x3ce1b5d to 0x631c50 2024-05-08T03:20:20.950 INFO:teuthology.orchestra.run.smithi177.stdout:3517 trunc from 0x631c50 to 0x42cf792 2024-05-08T03:20:20.967 INFO:teuthology.orchestra.run.smithi177.stdout:3518 writesame 0x63a4032 thru 0x63a732f data_size 0x3d(0x32fe bytes) 2024-05-08T03:20:20.967 INFO:teuthology.orchestra.run.smithi177.stdout:3519 punch from 0x38e1e35 to 0x38eec65, (0xce30 bytes) 2024-05-08T03:20:20.968 INFO:teuthology.orchestra.run.smithi177.stdout:3522 read 0x4663bcb thru 0x466d05c (0x9492 bytes) 2024-05-08T03:20:20.968 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.968 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.968 INFO:teuthology.orchestra.run.smithi177.stdout:3527 punch from 0x18705b7 to 0x187df08, (0xd951 bytes) 2024-05-08T03:20:20.968 INFO:teuthology.orchestra.run.smithi177.stdout:3528 writesame 0x575ed0a thru 0x5766f6e data_size 0x3(0x8265 bytes) 2024-05-08T03:20:20.975 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:20.979 INFO:teuthology.orchestra.run.smithi177.stdout:3530 trunc from 0x63a7330 to 0xa2f203c 2024-05-08T03:20:20.989 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:20.987+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:20.991 INFO:teuthology.orchestra.run.smithi177.stdout:3531 write 0xb1b09cc thru 0xb1b384b (0x2e80 bytes) 2024-05-08T03:20:20.993 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:20.993 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:20.993 INFO:teuthology.orchestra.run.smithi177.stdout:3534 write 0x9dce1ad thru 0x9dd3e70 (0x5cc4 bytes) 2024-05-08T03:20:20.996 INFO:teuthology.orchestra.run.smithi177.stdout:3535 punch from 0x30fd7fa to 0x31027c1, (0x4fc7 bytes) 2024-05-08T03:20:20.996 INFO:teuthology.orchestra.run.smithi177.stdout:3537 writesame 0x3d623b0 thru 0x3d63dce data_size 0x3(0x1a1f bytes) 2024-05-08T03:20:21.000 INFO:teuthology.orchestra.run.smithi177.stdout:3539 punch from 0x4f58409 to 0x4f67d1f, (0xf916 bytes) 2024-05-08T03:20:21.000 INFO:teuthology.orchestra.run.smithi177.stdout:3540 read 0x9b64d85 thru 0x9b6d11c (0x8398 bytes) 2024-05-08T03:20:21.000 INFO:teuthology.orchestra.run.smithi177.stdout:3543 write 0x86250b1 thru 0x8630e01 (0xbd51 bytes) 2024-05-08T03:20:21.002 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:21.005 INFO:teuthology.orchestra.run.smithi177.stdout:3546 write 0x6c74766 thru 0x6c80e45 (0xc6e0 bytes) 2024-05-08T03:20:21.008 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.009 INFO:teuthology.orchestra.run.smithi177.stdout:3548 trunc from 0xb1b384c to 0xc1104d0 2024-05-08T03:20:21.017 INFO:teuthology.orchestra.run.smithi177.stdout:3549 read 0x83aefec thru 0x83b3cdf (0x4cf4 bytes) 2024-05-08T03:20:21.017 INFO:teuthology.orchestra.run.smithi177.stdout:3550 punch from 0x19b99c to 0x1a4e8c, (0x94f0 bytes) 2024-05-08T03:20:21.017 INFO:teuthology.orchestra.run.smithi177.stdout:3552 read 0x1f833e thru 0x205330 (0xcff3 bytes) 2024-05-08T03:20:21.017 INFO:teuthology.orchestra.run.smithi177.stdout:3553 write 0x247677c thru 0x247c06f (0x58f4 bytes) 2024-05-08T03:20:21.019 INFO:teuthology.orchestra.run.smithi177.stdout:3554 writesame 0x4c41df2 thru 0x4c48a91 data_size 0x58(0x6ca0 bytes) 2024-05-08T03:20:21.033 INFO:teuthology.orchestra.run.smithi177.stdout:3555 trunc from 0xc1104d0 to 0x412c4d7 2024-05-08T03:20:21.072 INFO:teuthology.orchestra.run.smithi177.stdout:3556 writesame 0xa7da390 thru 0xa7e5793 data_size 0x29(0xb404 bytes) 2024-05-08T03:20:21.072 INFO:teuthology.orchestra.run.smithi177.stdout:3557 read 0x347f26c thru 0x347f406 (0x19b bytes) 2024-05-08T03:20:21.072 INFO:teuthology.orchestra.run.smithi177.stdout:3558 read 0x20f79e1 thru 0x2100c95 (0x92b5 bytes) 2024-05-08T03:20:21.072 INFO:teuthology.orchestra.run.smithi177.stdout:3559 write 0x84a4eac thru 0x84b4872 (0xf9c7 bytes) 2024-05-08T03:20:21.075 INFO:teuthology.orchestra.run.smithi177.stdout:3560 read 0x805fead thru 0x80697e3 (0x9937 bytes) 2024-05-08T03:20:21.078 INFO:teuthology.orchestra.run.smithi177.stdout:3561 trunc from 0xa7e5794 to 0xcd9cdd7 2024-05-08T03:20:21.082 INFO:teuthology.orchestra.run.smithi177.stdout:3564 write 0x9727e8a thru 0x97295b8 (0x172f bytes) 2024-05-08T03:20:21.086 INFO:teuthology.orchestra.run.smithi177.stdout:3565 read 0x4dd9adb thru 0x4de030e (0x6834 bytes) 2024-05-08T03:20:21.090 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:21.088+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:21.092 INFO:teuthology.orchestra.run.smithi177.stdout:3566 writesame 0xda0b35e thru 0xda1635a data_size 0x1(0xaffd bytes) 2024-05-08T03:20:21.104 INFO:teuthology.orchestra.run.smithi177.stdout:3567 trunc from 0xda1635b to 0x3ade7e5 2024-05-08T03:20:21.111 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6] out_osds: [4, 7, 0, 5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:21.112 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:21.112 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-08T03:20:21.112 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:20:21.121 INFO:teuthology.orchestra.run.smithi177.stdout:3569 write 0xa5c545 thru 0xa6b087 (0xeb43 bytes) 2024-05-08T03:20:21.124 INFO:teuthology.orchestra.run.smithi177.stdout:3570 read 0x32ec961 thru 0x32f4967 (0x8007 bytes) 2024-05-08T03:20:21.124 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.124 INFO:teuthology.orchestra.run.smithi177.stdout:3572 read 0x5b7bbe thru 0x5c2faa (0xb3ed bytes) 2024-05-08T03:20:21.131 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:21.131 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:21.137 INFO:teuthology.orchestra.run.smithi177.stdout:3573 write 0x957ee5c thru 0x9581b5b (0x2d00 bytes) 2024-05-08T03:20:21.141 INFO:teuthology.orchestra.run.smithi177.stdout:3574 read 0x53c5ae0 thru 0x53d1dae (0xc2cf bytes) 2024-05-08T03:20:21.141 INFO:teuthology.orchestra.run.smithi177.stdout:3575 write 0x19ac1c6 thru 0x19ad074 (0xeaf bytes) 2024-05-08T03:20:21.142 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:21.143 INFO:teuthology.orchestra.run.smithi177.stdout:3576 read 0x658add7 thru 0x658d6b3 (0x28dd bytes) 2024-05-08T03:20:21.144 INFO:teuthology.orchestra.run.smithi177.stdout:3577 writesame 0x31ff459 thru 0x31ffdc3 data_size 0x1(0x96b bytes) 2024-05-08T03:20:21.146 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.146 INFO:teuthology.orchestra.run.smithi177.stdout:3579 punch from 0x3a09738 to 0x3a0d37d, (0x3c45 bytes) 2024-05-08T03:20:21.146 INFO:teuthology.orchestra.run.smithi177.stdout:3580 write 0x22f253a thru 0x22fa9b2 (0x8479 bytes) 2024-05-08T03:20:21.148 INFO:teuthology.orchestra.run.smithi177.stdout:3581 write 0x3cb891 thru 0x3cbf6d (0x6dd bytes) 2024-05-08T03:20:21.150 INFO:teuthology.orchestra.run.smithi177.stdout:3583 read 0x1d56eed thru 0x1d57a24 (0xb38 bytes) 2024-05-08T03:20:21.150 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.153 INFO:teuthology.orchestra.run.smithi177.stdout:3585 trunc from 0x9581b5c to 0xbb9e594 2024-05-08T03:20:21.159 INFO:teuthology.orchestra.run.smithi177.stdout:3587 writesame 0x464d0b6 thru 0x465680a data_size 0x1(0x9755 bytes) 2024-05-08T03:20:21.180 INFO:teuthology.orchestra.run.smithi177.stdout:3589 punch from 0x48b5519 to 0x48b8530, (0x3017 bytes) 2024-05-08T03:20:21.180 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.180 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.180 INFO:teuthology.orchestra.run.smithi177.stdout:3593 write 0x6495d80 thru 0x649ed12 (0x8f93 bytes) 2024-05-08T03:20:21.184 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.184 INFO:teuthology.orchestra.run.smithi177.stdout:3599 punch from 0x7f380cc to 0x7f3f15f, (0x7093 bytes) 2024-05-08T03:20:21.184 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.190 INFO:teuthology.orchestra.run.smithi177.stdout:3602 writesame 0xbfb4b33 thru 0xbfbb9d7 data_size 0x1(0x6ea5 bytes) 2024-05-08T03:20:21.190 INFO:teuthology.orchestra.run.smithi177.stdout:3603 read 0x4c08554 thru 0x4c0c27c (0x3d29 bytes) 2024-05-08T03:20:21.191 INFO:teuthology.orchestra.run.smithi177.stdout:3605 read 0x91a17b2 thru 0x91a6f4a (0x5799 bytes) 2024-05-08T03:20:21.191 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:21.189+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:21.193 INFO:teuthology.orchestra.run.smithi177.stdout:3606 trunc from 0xbfbb9d8 to 0xde82fe1 2024-05-08T03:20:21.199 INFO:teuthology.orchestra.run.smithi177.stdout:3607 read 0x3cfcdc9 thru 0x3d062f5 (0x952d bytes) 2024-05-08T03:20:21.199 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.199 INFO:teuthology.orchestra.run.smithi177.stdout:3609 write 0x2d5c3de thru 0x2d6131d (0x4f40 bytes) 2024-05-08T03:20:21.203 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.203 INFO:teuthology.orchestra.run.smithi177.stdout:3611 read 0xa6ac526 thru 0xa6b0314 (0x3def bytes) 2024-05-08T03:20:21.203 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.203 INFO:teuthology.orchestra.run.smithi177.stdout:3613 punch from 0xa89bbf9 to 0xa8a1f43, (0x634a bytes) 2024-05-08T03:20:21.203 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.203 INFO:teuthology.orchestra.run.smithi177.stdout:3615 read 0x19ddd4d thru 0x19eccf9 (0xefad bytes) 2024-05-08T03:20:21.204 INFO:teuthology.orchestra.run.smithi177.stdout:3616 write 0x2ad3834 thru 0x2adda92 (0xa25f bytes) 2024-05-08T03:20:21.205 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.205 INFO:teuthology.orchestra.run.smithi177.stdout:3619 punch from 0x4c01afd to 0x4c0bc7a, (0xa17d bytes) 2024-05-08T03:20:21.205 INFO:teuthology.orchestra.run.smithi177.stdout:3620 writesame 0x1f8b8e4 thru 0x1f9b0c3 data_size 0x30(0xf7e0 bytes) 2024-05-08T03:20:21.208 INFO:teuthology.orchestra.run.smithi177.stdout:3621 trunc from 0xde82fe1 to 0xe8fc1e4 2024-05-08T03:20:21.214 INFO:teuthology.orchestra.run.smithi177.stdout:3622 read 0xdad33ba thru 0xdad62d7 (0x2f1e bytes) 2024-05-08T03:20:21.214 INFO:teuthology.orchestra.run.smithi177.stdout:3623 write 0xdc2b448 thru 0xdc2e2a9 (0x2e62 bytes) 2024-05-08T03:20:21.217 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.218 INFO:teuthology.orchestra.run.smithi177.stdout:3625 read 0x4891792 thru 0x489e166 (0xc9d5 bytes) 2024-05-08T03:20:21.218 INFO:teuthology.orchestra.run.smithi177.stdout:3626 read 0x9d0042c thru 0x9d0f16f (0xed44 bytes) 2024-05-08T03:20:21.218 INFO:teuthology.orchestra.run.smithi177.stdout:3627 read 0x80cfe50 thru 0x80d2271 (0x2422 bytes) 2024-05-08T03:20:21.218 INFO:teuthology.orchestra.run.smithi177.stdout:3628 read 0x81deb28 thru 0x81eeae4 (0xffbd bytes) 2024-05-08T03:20:21.218 INFO:teuthology.orchestra.run.smithi177.stdout:3629 read 0xc7f755a thru 0xc7fc4b8 (0x4f5f bytes) 2024-05-08T03:20:21.230 INFO:teuthology.orchestra.run.smithi177.stdout:3632 trunc from 0xe8fc1e4 to 0x3c1068b 2024-05-08T03:20:21.254 INFO:teuthology.orchestra.run.smithi177.stdout:3635 write 0x8922be5 thru 0x892d5f9 (0xaa15 bytes) 2024-05-08T03:20:21.257 INFO:teuthology.orchestra.run.smithi177.stdout:3636 write 0x3d4a4d9 thru 0x3d4fdb3 (0x58db bytes) 2024-05-08T03:20:21.259 INFO:teuthology.orchestra.run.smithi177.stdout:3638 punch from 0x27c67f5 to 0x27cf8ba, (0x90c5 bytes) 2024-05-08T03:20:21.260 INFO:teuthology.orchestra.run.smithi177.stdout:3639 read 0x639c1f3 thru 0x63a747f (0xb28d bytes) 2024-05-08T03:20:21.260 INFO:teuthology.orchestra.run.smithi177.stdout:3640 read 0x6ace206 thru 0x6acfd1b (0x1b16 bytes) 2024-05-08T03:20:21.260 INFO:teuthology.orchestra.run.smithi177.stdout:3641 writesame 0x388bbb8 thru 0x38945da data_size 0x1(0x8a23 bytes) 2024-05-08T03:20:21.269 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:21.269 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:21.279 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:21.280 INFO:teuthology.orchestra.run.smithi177.stdout:3643 punch from 0x8738628 to 0x87421c3, (0x9b9b bytes) 2024-05-08T03:20:21.280 INFO:teuthology.orchestra.run.smithi177.stdout:3645 read 0x4b56da thru 0x4be893 (0x91ba bytes) 2024-05-08T03:20:21.280 INFO:teuthology.orchestra.run.smithi177.stdout:3648 punch from 0x205c0b6 to 0x2064e0e, (0x8d58 bytes) 2024-05-08T03:20:21.280 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.281 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.294 INFO:teuthology.orchestra.run.smithi177.stdout:3653 write 0xe97de6b thru 0xe98a7f2 (0xc988 bytes) 2024-05-08T03:20:21.294 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:21.292+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:21.298 INFO:teuthology.orchestra.run.smithi177.stdout:3655 read 0x304c80d thru 0x3050e6f (0x4663 bytes) 2024-05-08T03:20:21.298 INFO:teuthology.orchestra.run.smithi177.stdout:3658 read 0x326a50d thru 0x3277323 (0xce17 bytes) 2024-05-08T03:20:21.299 INFO:teuthology.orchestra.run.smithi177.stdout:3659 read 0x84369e9 thru 0x843f418 (0x8a30 bytes) 2024-05-08T03:20:21.299 INFO:teuthology.orchestra.run.smithi177.stdout:3660 write 0x2c42479 thru 0x2c444b8 (0x2040 bytes) 2024-05-08T03:20:21.300 INFO:teuthology.orchestra.run.smithi177.stdout:3662 writesame 0x634890f thru 0x634fcde data_size 0x22(0x73d0 bytes) 2024-05-08T03:20:21.319 INFO:teuthology.orchestra.run.smithi177.stdout:3663 trunc from 0xe98a7f3 to 0x20c3055 2024-05-08T03:20:21.331 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.331 INFO:teuthology.orchestra.run.smithi177.stdout:3666 read 0xc72af0 thru 0xc7f8be (0xcdcf bytes) 2024-05-08T03:20:21.332 INFO:teuthology.orchestra.run.smithi177.stdout:3667 punch from 0x69628b to 0x6a0337, (0xa0ac bytes) 2024-05-08T03:20:21.332 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.340 INFO:teuthology.orchestra.run.smithi177.stdout:3669 write 0x4b646c6 thru 0x4b6f599 (0xaed4 bytes) 2024-05-08T03:20:21.343 INFO:teuthology.orchestra.run.smithi177.stdout:3671 read 0x437fbeb thru 0x438d19a (0xd5b0 bytes) 2024-05-08T03:20:21.343 INFO:teuthology.orchestra.run.smithi177.stdout:3672 read 0x1cb4c95 thru 0x1cb7a6c (0x2dd8 bytes) 2024-05-08T03:20:21.349 INFO:teuthology.orchestra.run.smithi177.stdout:3673 writesame 0x5f19676 thru 0x5f22536 data_size 0x1(0x8ec1 bytes) 2024-05-08T03:20:21.350 INFO:teuthology.orchestra.run.smithi177.stdout:3674 trunc from 0x5f22537 to 0x681aa4c 2024-05-08T03:20:21.355 INFO:teuthology.orchestra.run.smithi177.stdout:3675 write 0x3ccf08b thru 0x3cdcb2f (0xdaa5 bytes) 2024-05-08T03:20:21.369 INFO:teuthology.orchestra.run.smithi177.stdout:3677 write 0xaa5c0e7 thru 0xaa68332 (0xc24c bytes) 2024-05-08T03:20:21.372 INFO:teuthology.orchestra.run.smithi177.stdout:3678 write 0x397f24 thru 0x3a5581 (0xd65e bytes) 2024-05-08T03:20:21.374 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.374 INFO:teuthology.orchestra.run.smithi177.stdout:3682 write 0x9650be0 thru 0x96557e0 (0x4c01 bytes) 2024-05-08T03:20:21.377 INFO:teuthology.orchestra.run.smithi177.stdout:3683 read 0x6e5f843 thru 0x6e6588c (0x604a bytes) 2024-05-08T03:20:21.377 INFO:teuthology.orchestra.run.smithi177.stdout:3684 write 0x8d4d1da thru 0x8d57df5 (0xac1c bytes) 2024-05-08T03:20:21.380 INFO:teuthology.orchestra.run.smithi177.stdout:3685 read 0x2a733e1 thru 0x2a81b2a (0xe74a bytes) 2024-05-08T03:20:21.380 INFO:teuthology.orchestra.run.smithi177.stdout:3686 write 0x35aff6f thru 0x35b6fe2 (0x7074 bytes) 2024-05-08T03:20:21.383 INFO:teuthology.orchestra.run.smithi177.stdout:3687 write 0x1e32e60 thru 0x1e366a7 (0x3848 bytes) 2024-05-08T03:20:21.392 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:21.391+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:21.395 INFO:teuthology.orchestra.run.smithi177.stdout:3688 write 0xda9b7bc thru 0xdaa7626 (0xbe6b bytes) 2024-05-08T03:20:21.398 INFO:teuthology.orchestra.run.smithi177.stdout:3690 punch from 0x568b2e to 0x56cffc, (0x44ce bytes) 2024-05-08T03:20:21.404 INFO:teuthology.orchestra.run.smithi177.stdout:3692 writesame 0xe5e7394 thru 0xe5efd11 data_size 0x1(0x897e bytes) 2024-05-08T03:20:21.405 INFO:teuthology.orchestra.run.smithi177.stdout:3693 writesame 0x6b8eccd thru 0x6b9bf6f data_size 0x1(0xd2a3 bytes) 2024-05-08T03:20:21.416 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:21.416 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:21.425 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:21.434 INFO:teuthology.orchestra.run.smithi177.stdout:3694 punch from 0x712cdfe to 0x7137b48, (0xad4a bytes) 2024-05-08T03:20:21.434 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.434 INFO:teuthology.orchestra.run.smithi177.stdout:3698 writesame 0x7b9d8cd thru 0x7ba399c data_size 0x8(0x60d0 bytes) 2024-05-08T03:20:21.438 INFO:teuthology.orchestra.run.smithi177.stdout:3699 write 0xc9a89c0 thru 0xc9b3632 (0xac73 bytes) 2024-05-08T03:20:21.441 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.441 INFO:teuthology.orchestra.run.smithi177.stdout:3701 write 0xcde9cca thru 0xcdf2a1e (0x8d55 bytes) 2024-05-08T03:20:21.444 INFO:teuthology.orchestra.run.smithi177.stdout:3702 write 0x9705ba6 thru 0x970d034 (0x748f bytes) 2024-05-08T03:20:21.445 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.449 INFO:teuthology.orchestra.run.smithi177.stdout:3704 trunc from 0xe5efd12 to 0xd0a94b4 2024-05-08T03:20:21.455 INFO:teuthology.orchestra.run.smithi177.stdout:3706 read 0xb495eb8 thru 0xb49c39e (0x64e7 bytes) 2024-05-08T03:20:21.456 INFO:teuthology.orchestra.run.smithi177.stdout:3707 write 0xbad6372 thru 0xbada0ec (0x3d7b bytes) 2024-05-08T03:20:21.456 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:21.456 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:21.456 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 71, 2024-05-08T03:20:21.456 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:20:21.456 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:20:20.097104+0000", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:20:08.032979+0000", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:19:55.957014+0000", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5801984, 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "noscrub", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 5, 2024-05-08T03:20:21.457 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "luminous", 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:20:21.458 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:20:21.459 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:21.460 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:21.461 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:21.462 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.630000114440918, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.630000114440918, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.56999999284744263, 2024-05-08T03:20:21.463 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.5, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.5, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:21.464 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:21.465 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:21.466 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:21.467 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:21.468 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 6.9000000953674316, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 6.9000000953674316, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:21.469 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:21.470 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:21.471 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "70", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 29, 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 70, 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:21.472 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:21.473 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:21.474 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.8599998950958252, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.8599998950958252, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.56999999284744263, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.6299999952316284, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.6299999952316284, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.475 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.476 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 40, 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.477 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.478 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:21.481 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 60, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.484 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.485 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.486 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 61, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.488 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.489 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.490 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.491 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.492 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 56, 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.493 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.494 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 19, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.499 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.500 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:20:21.501 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:21.502 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 0, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 45, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 61, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:20:21.504 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1643429991", 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1643429991", 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1643429991", 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1643429991", 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: "exists" 2024-05-08T03:20:21.507 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 54, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.508 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.509 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.511 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:20:21.512 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:21.513 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:20:21.514 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.515 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.516 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:21.518 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:20:21.519 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:21.520 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:21.521 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 61 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:21.523 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [ 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "1.5", 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: 2, 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: 0 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.524 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.d", 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: 6 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [ 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.6", 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 1, 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.12", 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 3, 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.17", 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:21.526 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 3, 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:20:21.527 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [], 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [], 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:20:21.528 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:21.529 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:21.529 INFO:teuthology.orchestra.run.smithi177.stdout:3708 trunc from 0xd0a94b4 to 0x72c5673 2024-05-08T03:20:21.531 INFO:tasks.thrashosds.thrasher:j is {'epoch': 71, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:20:20.097104+0000', 'last_up_change': '2024-05-08T03:20:08.032979+0000', 'last_in_change': '2024-05-08T03:19:55.957014+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 0.5699999928474426, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.859999895095825, 'score_stable': 2.859999895095825, 'optimal_score': 0.5699999928474426, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 45, 'down_at': 61, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1643429991}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1643429991}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1643429991}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1643429991}]}, 'public_addr': '172.21.15.188:6817/1643429991', 'cluster_addr': '172.21.15.188:6819/1643429991', 'heartbeat_back_addr': '172.21.15.188:6823/1643429991', 'heartbeat_front_addr': '172.21.15.188:6821/1643429991', 'state': ['exists']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 61}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.5', 'osds': [2, 0]}, {'pgid': '3.d', 'osds': [3, 6]}], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '3.12', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.17', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:20:21.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:20:21.532 INFO:teuthology.orchestra.run.smithi177.stdout:3709 read 0x2efc6dc thru 0x2efed86 (0x26ab bytes) 2024-05-08T03:20:21.532 INFO:teuthology.orchestra.run.smithi177.stdout:3710 write 0xd89f2d3 thru 0xd89f645 (0x373 bytes) 2024-05-08T03:20:21.532 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:3712 writesame 0x426eeee thru 0x426fbac data_size 0x1(0xcbf bytes) 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:3713 write 0x20b2ed thru 0x218922 (0xd636 bytes) 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:3716 write 0xa46ea15 thru 0xa47244c (0x3a38 bytes) 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:3717 write 0x2fb05d2 thru 0x2fbb987 (0xb3b6 bytes) 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:3719 trunc from 0xd89f646 to 0x778896e 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.533 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.533 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:21.528+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:21.538 INFO:teuthology.orchestra.run.smithi177.stdout:3722 writesame 0xd3f0cd3 thru 0xd4009d6 data_size 0x2(0xfd04 bytes) 2024-05-08T03:20:21.538 INFO:teuthology.orchestra.run.smithi177.stdout:3723 write 0xb4b027c thru 0xb4b2b10 (0x2895 bytes) 2024-05-08T03:20:21.541 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.541 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.541 INFO:teuthology.orchestra.run.smithi177.stdout:3729 write 0xc02dd5f thru 0xc02eebf (0x1161 bytes) 2024-05-08T03:20:21.552 INFO:teuthology.orchestra.run.smithi177.stdout:3730 write 0xec5724c thru 0xec6362b (0xc3e0 bytes) 2024-05-08T03:20:21.556 INFO:teuthology.orchestra.run.smithi177.stdout:3731 writesame 0x4a0259 thru 0x4acc58 data_size 0x80(0xca00 bytes) 2024-05-08T03:20:21.557 INFO:teuthology.orchestra.run.smithi177.stdout:3732 write 0x5ac0735 thru 0x5ac931f (0x8beb bytes) 2024-05-08T03:20:21.559 INFO:teuthology.orchestra.run.smithi177.stdout:3733 read 0xd58ea86 thru 0xd598f31 (0xa4ac bytes) 2024-05-08T03:20:21.559 INFO:teuthology.orchestra.run.smithi177.stdout:3734 read 0x1d5fd37 thru 0x1d6d06e (0xd338 bytes) 2024-05-08T03:20:21.560 INFO:teuthology.orchestra.run.smithi177.stdout:3736 read 0x9427346 thru 0x94349a9 (0xd664 bytes) 2024-05-08T03:20:21.560 INFO:teuthology.orchestra.run.smithi177.stdout:3737 read 0xb836700 thru 0xb845818 (0xf119 bytes) 2024-05-08T03:20:21.560 INFO:teuthology.orchestra.run.smithi177.stdout:3740 write 0x963f4dc thru 0x964e5c6 (0xf0eb bytes) 2024-05-08T03:20:21.563 INFO:teuthology.orchestra.run.smithi177.stdout:3741 read 0x414cfe8 thru 0x415c60d (0xf626 bytes) 2024-05-08T03:20:21.563 INFO:teuthology.orchestra.run.smithi177.stdout:3742 read 0x67b0d06 thru 0x67bb196 (0xa491 bytes) 2024-05-08T03:20:21.564 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:21.564 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:21.580 INFO:teuthology.orchestra.run.smithi177.stdout:3744 trunc from 0xec6362c to 0x112e205 2024-05-08T03:20:21.581 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:21.598 INFO:teuthology.orchestra.run.smithi177.stdout:3745 read 0x503985 thru 0x512a30 (0xf0ac bytes) 2024-05-08T03:20:21.598 INFO:teuthology.orchestra.run.smithi177.stdout:3746 read 0x9893b4 thru 0x9963c5 (0xd012 bytes) 2024-05-08T03:20:21.607 INFO:teuthology.orchestra.run.smithi177.stdout:3747 write 0x3ce06b0 thru 0x3ce246e (0x1dbf bytes) 2024-05-08T03:20:21.611 INFO:teuthology.orchestra.run.smithi177.stdout:3748 read 0x37592c1 thru 0x37662d4 (0xd014 bytes) 2024-05-08T03:20:21.623 INFO:teuthology.orchestra.run.smithi177.stdout:3749 write 0x889b437 thru 0x889e25c (0x2e26 bytes) 2024-05-08T03:20:21.626 INFO:teuthology.orchestra.run.smithi177.stdout:3750 read 0x259422b thru 0x259e170 (0x9f46 bytes) 2024-05-08T03:20:21.630 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:21.629+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:21.636 INFO:teuthology.orchestra.run.smithi177.stdout:3754 writesame 0xc04b97f thru 0xc0583a9 data_size 0xb(0xca2b bytes) 2024-05-08T03:20:21.636 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.636 INFO:teuthology.orchestra.run.smithi177.stdout:3758 write 0xbfdeba0 thru 0xbfecd2a (0xe18b bytes) 2024-05-08T03:20:21.639 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.639 INFO:teuthology.orchestra.run.smithi177.stdout:3761 punch from 0x8a032e9 to 0x8a10ff5, (0xdd0c bytes) 2024-05-08T03:20:21.639 INFO:teuthology.orchestra.run.smithi177.stdout:3765 write 0x256e009 thru 0x257a71e (0xc716 bytes) 2024-05-08T03:20:21.649 INFO:teuthology.orchestra.run.smithi177.stdout:3766 writesame 0xe814e40 thru 0xe818103 data_size 0x4c(0x32c4 bytes) 2024-05-08T03:20:21.649 INFO:teuthology.orchestra.run.smithi177.stdout:3767 write 0x659da79 thru 0x65a6896 (0x8e1e bytes) 2024-05-08T03:20:21.651 INFO:teuthology.orchestra.run.smithi177.stdout:3769 write 0x51237bc thru 0x5127305 (0x3b4a bytes) 2024-05-08T03:20:21.654 INFO:teuthology.orchestra.run.smithi177.stdout:3770 write 0x7c404b2 thru 0x7c4a5b6 (0xa105 bytes) 2024-05-08T03:20:21.655 INFO:teuthology.orchestra.run.smithi177.stdout:3773 write 0xbc4b0b3 thru 0xbc4cba4 (0x1af2 bytes) 2024-05-08T03:20:21.671 INFO:teuthology.orchestra.run.smithi177.stdout:3776 trunc from 0xe818104 to 0x45a34ac 2024-05-08T03:20:21.686 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.686 INFO:teuthology.orchestra.run.smithi177.stdout:3780 read 0x25d8e45 thru 0x25e2d73 (0x9f2f bytes) 2024-05-08T03:20:21.686 INFO:teuthology.orchestra.run.smithi177.stdout:3781 read 0x20105c3 thru 0x2010e58 (0x896 bytes) 2024-05-08T03:20:21.686 INFO:teuthology.orchestra.run.smithi177.stdout:3782 write 0x306fabc thru 0x307797b (0x7ec0 bytes) 2024-05-08T03:20:21.688 INFO:teuthology.orchestra.run.smithi177.stdout:3783 punch from 0x20c115e to 0x20c6c69, (0x5b0b bytes) 2024-05-08T03:20:21.697 INFO:teuthology.orchestra.run.smithi177.stdout:3784 trunc from 0x45a34ac to 0xc538675 2024-05-08T03:20:21.712 INFO:teuthology.orchestra.run.smithi177.stdout:3785 trunc from 0xc538675 to 0x572c5f2 2024-05-08T03:20:21.718 INFO:teuthology.orchestra.run.smithi177.stdout:3786 read 0x51d863a thru 0x51db83a (0x3201 bytes) 2024-05-08T03:20:21.719 INFO:teuthology.orchestra.run.smithi177.stdout:3787 read 0x4c97742 thru 0x4c9cb1d (0x53dc bytes) 2024-05-08T03:20:21.719 INFO:teuthology.orchestra.run.smithi177.stdout:3788 punch from 0x2771696 to 0x277fa97, (0xe401 bytes) 2024-05-08T03:20:21.719 INFO:teuthology.orchestra.run.smithi177.stdout:3789 read 0x315a571 thru 0x315d6ec (0x317c bytes) 2024-05-08T03:20:21.724 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:21.724 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:21.730 INFO:teuthology.orchestra.run.smithi177.stdout:3790 write 0xa631223 thru 0xa63fe64 (0xec42 bytes) 2024-05-08T03:20:21.731 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:21.729+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:21.734 INFO:teuthology.orchestra.run.smithi177.stdout:3791 write 0x8e8ce05 thru 0x8e99633 (0xc82f bytes) 2024-05-08T03:20:21.735 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:21.742 INFO:teuthology.orchestra.run.smithi177.stdout:3792 writesame 0xa9fdd1b thru 0xaa0adaf data_size 0x33(0xd095 bytes) 2024-05-08T03:20:21.742 INFO:teuthology.orchestra.run.smithi177.stdout:3793 punch from 0x417d58c to 0x41815cc, (0x4040 bytes) 2024-05-08T03:20:21.744 INFO:teuthology.orchestra.run.smithi177.stdout:3794 trunc from 0xaa0adb0 to 0x87d724f 2024-05-08T03:20:21.755 INFO:teuthology.orchestra.run.smithi177.stdout:3795 punch from 0x33ad678 to 0x33b4da6, (0x772e bytes) 2024-05-08T03:20:21.755 INFO:teuthology.orchestra.run.smithi177.stdout:3796 read 0x14f82b9 thru 0x150271e (0xa466 bytes) 2024-05-08T03:20:21.765 INFO:teuthology.orchestra.run.smithi177.stdout:3797 write 0xd157c61 thru 0xd15d7c1 (0x5b61 bytes) 2024-05-08T03:20:21.769 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.769 INFO:teuthology.orchestra.run.smithi177.stdout:3799 punch from 0x4e551d1 to 0x4e63b6e, (0xe99d bytes) 2024-05-08T03:20:21.769 INFO:teuthology.orchestra.run.smithi177.stdout:3800 punch from 0xfd4266 to 0xfe2a15, (0xe7af bytes) 2024-05-08T03:20:21.769 INFO:teuthology.orchestra.run.smithi177.stdout:3801 read 0xa72bb5f thru 0xa72e42b (0x28cd bytes) 2024-05-08T03:20:21.769 INFO:teuthology.orchestra.run.smithi177.stdout:3803 writesame 0x8db284 thru 0x8e367b data_size 0x67(0x83f8 bytes) 2024-05-08T03:20:21.770 INFO:teuthology.orchestra.run.smithi177.stdout:3804 write 0x13726ac thru 0x137d4af (0xae04 bytes) 2024-05-08T03:20:21.771 INFO:teuthology.orchestra.run.smithi177.stdout:truncating to largest ever: 0xee31ee6 2024-05-08T03:20:21.774 INFO:teuthology.orchestra.run.smithi177.stdout:3805 trunc from 0xd15d7c2 to 0xee31ee6 2024-05-08T03:20:21.780 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.780 INFO:teuthology.orchestra.run.smithi177.stdout:3807 write 0xc9aeeb1 thru 0xc9b45ef (0x573f bytes) 2024-05-08T03:20:21.795 INFO:teuthology.orchestra.run.smithi177.stdout:3808 trunc from 0xee31ee6 to 0x451436d 2024-05-08T03:20:21.816 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.816 INFO:teuthology.orchestra.run.smithi177.stdout:3811 punch from 0x43f4314 to 0x43f6938, (0x2624 bytes) 2024-05-08T03:20:21.831 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:21.829+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:21.833 INFO:teuthology.orchestra.run.smithi177.stdout:3812 write 0xdb3ac50 thru 0xdb42a75 (0x7e26 bytes) 2024-05-08T03:20:21.836 INFO:teuthology.orchestra.run.smithi177.stdout:3813 write 0x4dcef70 thru 0x4dd4abf (0x5b50 bytes) 2024-05-08T03:20:21.839 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.839 INFO:teuthology.orchestra.run.smithi177.stdout:3815 write 0x24682d5 thru 0x2469544 (0x1270 bytes) 2024-05-08T03:20:21.841 INFO:teuthology.orchestra.run.smithi177.stdout:3816 writesame 0x6231cfd thru 0x623ae89 data_size 0x1(0x918d bytes) 2024-05-08T03:20:21.862 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:21.862 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:21.866 INFO:teuthology.orchestra.run.smithi177.stdout:3817 write 0x26ae866 thru 0x26b4341 (0x5adc bytes) 2024-05-08T03:20:21.868 INFO:teuthology.orchestra.run.smithi177.stdout:3819 read 0x3e65ae0 thru 0x3e7106d (0xb58e bytes) 2024-05-08T03:20:21.868 INFO:teuthology.orchestra.run.smithi177.stdout:3820 read 0xba441f9 thru 0xba53992 (0xf79a bytes) 2024-05-08T03:20:21.868 INFO:teuthology.orchestra.run.smithi177.stdout:3821 punch from 0x7eac8bb to 0x7eb134e, (0x4a93 bytes) 2024-05-08T03:20:21.872 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:21.873 INFO:teuthology.orchestra.run.smithi177.stdout:3822 write 0xdc88521 thru 0xdc96044 (0xdb24 bytes) 2024-05-08T03:20:21.878 INFO:teuthology.orchestra.run.smithi177.stdout:3823 read 0xa5c3106 thru 0xa5ccfec (0x9ee7 bytes) 2024-05-08T03:20:21.878 INFO:teuthology.orchestra.run.smithi177.stdout:3824 writesame 0x9a34902 thru 0x9a3d5e1 data_size 0x38(0x8ce0 bytes) 2024-05-08T03:20:21.881 INFO:teuthology.orchestra.run.smithi177.stdout:3825 punch from 0x12dade9 to 0x12e1230, (0x6447 bytes) 2024-05-08T03:20:21.881 INFO:teuthology.orchestra.run.smithi177.stdout:3826 punch from 0xb2e7a35 to 0xb2ef880, (0x7e4b bytes) 2024-05-08T03:20:21.881 INFO:teuthology.orchestra.run.smithi177.stdout:3827 punch from 0x72aa178 to 0x72b2b72, (0x89fa bytes) 2024-05-08T03:20:21.881 INFO:teuthology.orchestra.run.smithi177.stdout:3828 write 0xa7b2db2 thru 0xa7bebce (0xbe1d bytes) 2024-05-08T03:20:21.882 INFO:teuthology.orchestra.run.smithi177.stdout:3833 read 0xae151bb thru 0xae20198 (0xafde bytes) 2024-05-08T03:20:21.882 INFO:teuthology.orchestra.run.smithi177.stdout:3834 read 0xc34d3f6 thru 0xc353e37 (0x6a42 bytes) 2024-05-08T03:20:21.883 INFO:teuthology.orchestra.run.smithi177.stdout:3836 write 0x4b2d21e thru 0x4b36d8f (0x9b72 bytes) 2024-05-08T03:20:21.886 INFO:teuthology.orchestra.run.smithi177.stdout:3838 read 0x78aa114 thru 0x78b5770 (0xb65d bytes) 2024-05-08T03:20:21.886 INFO:teuthology.orchestra.run.smithi177.stdout:3839 write 0x454c109 thru 0x45568b8 (0xa7b0 bytes) 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:3841 read 0x702a097 thru 0x703220c (0x8176 bytes) 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:3844 read 0x6860a08 thru 0x687066f (0xfc68 bytes) 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:3845 read 0x61e56e2 thru 0x61f54ec (0xfe0b bytes) 2024-05-08T03:20:21.888 INFO:teuthology.orchestra.run.smithi177.stdout:3846 write 0x7450bb8 thru 0x7454235 (0x367e bytes) 2024-05-08T03:20:21.891 INFO:teuthology.orchestra.run.smithi177.stdout:3847 punch from 0xbf2d148 to 0xbf3b2a4, (0xe15c bytes) 2024-05-08T03:20:21.891 INFO:teuthology.orchestra.run.smithi177.stdout:3848 punch from 0x383ed68 to 0x3845abf, (0x6d57 bytes) 2024-05-08T03:20:21.891 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.891 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.891 INFO:teuthology.orchestra.run.smithi177.stdout:3852 punch from 0xc6072f7 to 0xc60fbb7, (0x88c0 bytes) 2024-05-08T03:20:21.891 INFO:teuthology.orchestra.run.smithi177.stdout:3855 write 0x4bc2264 thru 0x4bcbf76 (0x9d13 bytes) 2024-05-08T03:20:21.893 INFO:teuthology.orchestra.run.smithi177.stdout:3856 write 0x853f794 thru 0x8549477 (0x9ce4 bytes) 2024-05-08T03:20:21.900 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:21.901 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:20:21.907 INFO:teuthology.orchestra.run.smithi177.stdout:3857 trunc from 0xdc96045 to 0x48d3f8f 2024-05-08T03:20:21.921 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-05-08T03:20:20.184369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198748698,"num_objects":1190,"num_object_clones":229,"num_object_copies":2380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1190,"num_whiteouts":418,"num_read":74487,"num_read_kb":608581,"num_write":9677,"num_write_kb":433642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1135,"num_bytes_recovered":161555472,"num_keys_recovered":229,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7851,"ondisk_log_size":7851,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":783020,"kb_used_data":637756,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":376704340,"statfs":{"total":386547056640,"available":385745244160,"internally_reserved":0,"allocated":653062144,"data_stored":957371476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6377,"internal_metadata":144697111},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":156732447,"num_objects":114,"num_object_clones":8,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":32,"num_read":24024,"num_read_kb":157352,"num_write":1701,"num_write_kb":54457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000557"},"pg_stats":[{"pgid":"3.1f","version":"66'137","reported_seq":1017,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.126255+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:20:16.126255+0000","last_peered":"2024-05-08T03:20:16.126255+0000","last_clean":"2024-05-08T03:20:16.126255+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:20:16.126255+0000","last_undegraded":"2024-05-08T03:20:16.126255+0000","last_fullsized":"2024-05-08T03:20:16.126255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:59:24.418622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":36347611,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":1159,"num_read_kb":7160,"num_write":166,"num_write_kb":6498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"66'126","reported_seq":534,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.094054+0000","last_change":"2024-05-08T03:19:22.850001+0000","last_active":"2024-05-08T03:20:16.094054+0000","last_peered":"2024-05-08T03:20:16.094054+0000","last_clean":"2024-05-08T03:20:16.094054+0000","last_became_active":"2024-05-08T03:19:20.773128+0000","last_became_peered":"2024-05-08T03:19:20.773128+0000","last_unstale":"2024-05-08T03:20:16.094054+0000","last_undegraded":"2024-05-08T03:20:16.094054+0000","last_fullsized":"2024-05-08T03:20:16.094054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:49.718815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":21714433,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":387,"num_read_kb":765,"num_write":122,"num_write_kb":1032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"66'287","reported_seq":3628,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096227+0000","last_change":"2024-05-08T03:20:03.022224+0000","last_active":"2024-05-08T03:20:16.096227+0000","last_peered":"2024-05-08T03:20:16.096227+0000","last_clean":"2024-05-08T03:20:16.096227+0000","last_became_active":"2024-05-08T03:20:03.021608+0000","last_became_peered":"2024-05-08T03:20:03.021608+0000","last_unstale":"2024-05-08T03:20:16.096227+0000","last_undegraded":"2024-05-08T03:20:16.096227+0000","last_fullsized":"2024-05-08T03:20:16.096227+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:56:41.356165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":54686352,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":4063,"num_read_kb":20089,"num_write":343,"num_write_kb":7039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":278351,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"67'229","reported_seq":1914,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.157235+0000","last_change":"2024-05-08T03:19:59.997205+0000","last_active":"2024-05-08T03:20:16.157235+0000","last_peered":"2024-05-08T03:20:16.157235+0000","last_clean":"2024-05-08T03:20:16.157235+0000","last_became_active":"2024-05-08T03:19:59.996571+0000","last_became_peered":"2024-05-08T03:19:59.996571+0000","last_unstale":"2024-05-08T03:20:16.157235+0000","last_undegraded":"2024-05-08T03:20:16.157235+0000","last_fullsized":"2024-05-08T03:20:16.157235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:39.711479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":41071967,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2082,"num_read_kb":41637,"num_write":307,"num_write_kb":38513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":1994536,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"66'274","reported_seq":1627,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096427+0000","last_change":"2024-05-08T03:19:51.215870+0000","last_active":"2024-05-08T03:20:16.096427+0000","last_peered":"2024-05-08T03:20:16.096427+0000","last_clean":"2024-05-08T03:20:16.096427+0000","last_became_active":"2024-05-08T03:19:50.935675+0000","last_became_peered":"2024-05-08T03:19:50.935675+0000","last_unstale":"2024-05-08T03:20:16.096427+0000","last_undegraded":"2024-05-08T03:20:16.096427+0000","last_fullsized":"2024-05-08T03:20:16.096427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:00.243757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":26883872,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2264,"num_read_kb":4307,"num_write":363,"num_write_kb":8457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2376803,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"67'372","reported_seq":2657,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.202456+0000","last_change":"2024-05-08T03:19:31.854145+0000","last_active":"2024-05-08T03:20:16.202456+0000","last_peered":"2024-05-08T03:20:16.202456+0000","last_clean":"2024-05-08T03:20:16.202456+0000","last_became_active":"2024-05-08T03:19:31.830295+0000","last_became_peered":"2024-05-08T03:19:31.830295+0000","last_unstale":"2024-05-08T03:20:16.202456+0000","last_undegraded":"2024-05-08T03:20:16.202456+0000","last_fullsized":"2024-05-08T03:20:16.202456+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:44.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":47345274,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3163,"num_read_kb":34273,"num_write":545,"num_write_kb":41765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":91773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"67'148","reported_seq":1329,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.122694+0000","last_change":"2024-05-08T03:19:22.779129+0000","last_active":"2024-05-08T03:20:16.122694+0000","last_peered":"2024-05-08T03:20:16.122694+0000","last_clean":"2024-05-08T03:20:16.122694+0000","last_became_active":"2024-05-08T03:19:20.775300+0000","last_became_peered":"2024-05-08T03:19:20.775300+0000","last_unstale":"2024-05-08T03:20:16.122694+0000","last_undegraded":"2024-05-08T03:20:16.122694+0000","last_fullsized":"2024-05-08T03:20:16.122694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:08.745847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":38418399,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":1116,"num_read_kb":4418,"num_write":142,"num_write_kb":5170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"67'267","reported_seq":5420,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.153649+0000","last_change":"2024-05-08T03:19:53.957566+0000","last_active":"2024-05-08T03:20:16.153649+0000","last_peered":"2024-05-08T03:20:16.153649+0000","last_clean":"2024-05-08T03:20:16.153649+0000","last_became_active":"2024-05-08T03:19:53.957072+0000","last_became_peered":"2024-05-08T03:19:53.957072+0000","last_unstale":"2024-05-08T03:20:16.153649+0000","last_undegraded":"2024-05-08T03:20:16.153649+0000","last_fullsized":"2024-05-08T03:20:16.153649+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:07:20.704153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":63185944,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":6171,"num_read_kb":39963,"num_write":269,"num_write_kb":8738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":6126489,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"66'146","reported_seq":2292,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093946+0000","last_change":"2024-05-08T03:20:06.164893+0000","last_active":"2024-05-08T03:20:16.093946+0000","last_peered":"2024-05-08T03:20:16.093946+0000","last_clean":"2024-05-08T03:20:16.093946+0000","last_became_active":"2024-05-08T03:20:06.035685+0000","last_became_peered":"2024-05-08T03:20:06.035685+0000","last_unstale":"2024-05-08T03:20:16.093946+0000","last_undegraded":"2024-05-08T03:20:16.093946+0000","last_fullsized":"2024-05-08T03:20:16.093946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:05.761803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":24815839,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":2165,"num_read_kb":17490,"num_write":147,"num_write_kb":2115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":192270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"66'99","reported_seq":1744,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096173+0000","last_change":"2024-05-08T03:19:57.081242+0000","last_active":"2024-05-08T03:20:16.096173+0000","last_peered":"2024-05-08T03:20:16.096173+0000","last_clean":"2024-05-08T03:20:16.096173+0000","last_became_active":"2024-05-08T03:19:56.989658+0000","last_became_peered":"2024-05-08T03:19:56.989658+0000","last_unstale":"2024-05-08T03:20:16.096173+0000","last_undegraded":"2024-05-08T03:20:16.096173+0000","last_fullsized":"2024-05-08T03:20:16.096173+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:23.352523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":75411156,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":1476,"num_read_kb":43618,"num_write":95,"num_write_kb":39103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":8790475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"67'166","reported_seq":3572,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.181428+0000","last_change":"2024-05-08T03:19:50.996510+0000","last_active":"2024-05-08T03:20:16.181428+0000","last_peered":"2024-05-08T03:20:16.181428+0000","last_clean":"2024-05-08T03:20:16.181428+0000","last_became_active":"2024-05-08T03:19:50.927921+0000","last_became_peered":"2024-05-08T03:19:50.927921+0000","last_unstale":"2024-05-08T03:20:16.181428+0000","last_undegraded":"2024-05-08T03:20:16.181428+0000","last_fullsized":"2024-05-08T03:20:16.181428+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:23.592209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":43858616,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3526,"num_read_kb":23282,"num_write":149,"num_write_kb":4244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":268258,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"66'408","reported_seq":1494,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093568+0000","last_change":"2024-05-08T03:20:03.022563+0000","last_active":"2024-05-08T03:20:16.093568+0000","last_peered":"2024-05-08T03:20:16.093568+0000","last_clean":"2024-05-08T03:20:16.093568+0000","last_became_active":"2024-05-08T03:20:03.021688+0000","last_became_peered":"2024-05-08T03:20:03.021688+0000","last_unstale":"2024-05-08T03:20:16.093568+0000","last_undegraded":"2024-05-08T03:20:16.093568+0000","last_fullsized":"2024-05-08T03:20:16.093568+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:17:55.272395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":12113452,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":2619,"num_read_kb":9922,"num_write":568,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":135734,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.094332+0000","last_change":"2024-05-08T03:20:03.022786+0000","last_active":"2024-05-08T03:20:16.094332+0000","last_peered":"2024-05-08T03:20:16.094332+0000","last_clean":"2024-05-08T03:20:16.094332+0000","last_became_active":"2024-05-08T03:20:03.022530+0000","last_became_peered":"2024-05-08T03:20:03.022530+0000","last_unstale":"2024-05-08T03:20:16.094332+0000","last_undegraded":"2024-05-08T03:20:16.094332+0000","last_fullsized":"2024-05-08T03:20:16.094332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:59.431605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"67'148","reported_seq":640,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.207651+0000","last_change":"2024-05-08T03:19:57.064707+0000","last_active":"2024-05-08T03:20:16.207651+0000","last_peered":"2024-05-08T03:20:16.207651+0000","last_clean":"2024-05-08T03:20:16.207651+0000","last_became_active":"2024-05-08T03:19:56.989832+0000","last_became_peered":"2024-05-08T03:19:56.989832+0000","last_unstale":"2024-05-08T03:20:16.207651+0000","last_undegraded":"2024-05-08T03:20:16.207651+0000","last_fullsized":"2024-05-08T03:20:16.207651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:26:06.555126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":8547557,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":360,"num_read_kb":3795,"num_write":141,"num_write_kb":2108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":208697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":98,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093965+0000","last_change":"2024-05-08T03:20:03.023919+0000","last_active":"2024-05-08T03:20:16.093965+0000","last_peered":"2024-05-08T03:20:16.093965+0000","last_clean":"2024-05-08T03:20:16.093965+0000","last_became_active":"2024-05-08T03:20:03.023611+0000","last_became_peered":"2024-05-08T03:20:03.023611+0000","last_unstale":"2024-05-08T03:20:16.093965+0000","last_undegraded":"2024-05-08T03:20:16.093965+0000","last_fullsized":"2024-05-08T03:20:16.093965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:04:35.966897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"66'409","reported_seq":1774,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096404+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:20:16.096404+0000","last_peered":"2024-05-08T03:20:16.096404+0000","last_clean":"2024-05-08T03:20:16.096404+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:20:16.096404+0000","last_undegraded":"2024-05-08T03:20:16.096404+0000","last_fullsized":"2024-05-08T03:20:16.096404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:10.484758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":22985992,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":3227,"num_read_kb":26388,"num_write":543,"num_write_kb":15916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.094185+0000","last_change":"2024-05-08T03:19:31.832510+0000","last_active":"2024-05-08T03:20:16.094185+0000","last_peered":"2024-05-08T03:20:16.094185+0000","last_clean":"2024-05-08T03:20:16.094185+0000","last_became_active":"2024-05-08T03:19:31.832151+0000","last_became_peered":"2024-05-08T03:19:31.832151+0000","last_unstale":"2024-05-08T03:20:16.094185+0000","last_undegraded":"2024-05-08T03:20:16.094185+0000","last_fullsized":"2024-05-08T03:20:16.094185+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:31.808173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"67'146","reported_seq":1960,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.212253+0000","last_change":"2024-05-08T03:19:57.108001+0000","last_active":"2024-05-08T03:20:16.212253+0000","last_peered":"2024-05-08T03:20:16.212253+0000","last_clean":"2024-05-08T03:20:16.212253+0000","last_became_active":"2024-05-08T03:19:56.991735+0000","last_became_peered":"2024-05-08T03:19:56.991735+0000","last_unstale":"2024-05-08T03:20:16.212253+0000","last_undegraded":"2024-05-08T03:20:16.212253+0000","last_fullsized":"2024-05-08T03:20:16.212253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:50:07.438086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":7102698,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":1706,"num_read_kb":7723,"num_write":151,"num_write_kb":2785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":78741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093940+0000","last_change":"2024-05-08T03:19:56.989000+0000","last_active":"2024-05-08T03:20:16.093940+0000","last_peered":"2024-05-08T03:20:16.093940+0000","last_clean":"2024-05-08T03:20:16.093940+0000","last_became_active":"2024-05-08T03:19:56.988667+0000","last_became_peered":"2024-05-08T03:19:56.988667+0000","last_unstale":"2024-05-08T03:20:16.093940+0000","last_undegraded":"2024-05-08T03:20:16.093940+0000","last_fullsized":"2024-05-08T03:20:16.093940+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:33.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"67'640","reported_seq":2800,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.185554+0000","last_change":"2024-05-08T03:20:09.057346+0000","last_active":"2024-05-08T03:20:16.185554+0000","last_peered":"2024-05-08T03:20:16.185554+0000","last_clean":"2024-05-08T03:20:16.185554+0000","last_became_active":"2024-05-08T03:20:09.056416+0000","last_became_peered":"2024-05-08T03:20:09.056416+0000","last_unstale":"2024-05-08T03:20:16.185554+0000","last_undegraded":"2024-05-08T03:20:16.185554+0000","last_fullsized":"2024-05-08T03:20:16.185554+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:08.985127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":27823626,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":4490,"num_read_kb":26509,"num_write":1043,"num_write_kb":29708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":30566047,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"67'111","reported_seq":990,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.226757+0000","last_change":"2024-05-08T03:19:57.175539+0000","last_active":"2024-05-08T03:20:16.226757+0000","last_peered":"2024-05-08T03:20:16.226757+0000","last_clean":"2024-05-08T03:20:16.226757+0000","last_became_active":"2024-05-08T03:19:56.994286+0000","last_became_peered":"2024-05-08T03:19:56.994286+0000","last_unstale":"2024-05-08T03:20:16.226757+0000","last_undegraded":"2024-05-08T03:20:16.226757+0000","last_fullsized":"2024-05-08T03:20:16.226757+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:48:20.710190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":11368062,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":675,"num_read_kb":5905,"num_write":102,"num_write_kb":4759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1928876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":244,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096421+0000","last_change":"2024-05-08T03:19:53.957635+0000","last_active":"2024-05-08T03:20:16.096421+0000","last_peered":"2024-05-08T03:20:16.096421+0000","last_clean":"2024-05-08T03:20:16.096421+0000","last_became_active":"2024-05-08T03:19:53.957252+0000","last_became_peered":"2024-05-08T03:19:53.957252+0000","last_unstale":"2024-05-08T03:20:16.096421+0000","last_undegraded":"2024-05-08T03:20:16.096421+0000","last_fullsized":"2024-05-08T03:20:16.096421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:27.307894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096590+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:20:16.096590+0000","last_peered":"2024-05-08T03:20:16.096590+0000","last_clean":"2024-05-08T03:20:16.096590+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:20:16.096590+0000","last_undegraded":"2024-05-08T03:20:16.096590+0000","last_fullsized":"2024-05-08T03:20:16.096590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"67'143","reported_seq":919,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.145792+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:20:16.145792+0000","last_peered":"2024-05-08T03:20:16.145792+0000","last_clean":"2024-05-08T03:20:16.145792+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:20:16.145792+0000","last_undegraded":"2024-05-08T03:20:16.145792+0000","last_fullsized":"2024-05-08T03:20:16.145792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:10.369244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":17769947,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":1221,"num_read_kb":2931,"num_write":128,"num_write_kb":1885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":60,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093679+0000","last_change":"2024-05-08T03:19:56.983410+0000","last_active":"2024-05-08T03:20:16.093679+0000","last_peered":"2024-05-08T03:20:16.093679+0000","last_clean":"2024-05-08T03:20:16.093679+0000","last_became_active":"2024-05-08T03:19:56.983063+0000","last_became_peered":"2024-05-08T03:19:56.983063+0000","last_unstale":"2024-05-08T03:20:16.093679+0000","last_undegraded":"2024-05-08T03:20:16.093679+0000","last_fullsized":"2024-05-08T03:20:16.093679+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:54:24.071706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"65'366","reported_seq":3454,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096277+0000","last_change":"2024-05-08T03:20:03.022688+0000","last_active":"2024-05-08T03:20:16.096277+0000","last_peered":"2024-05-08T03:20:16.096277+0000","last_clean":"2024-05-08T03:20:16.096277+0000","last_became_active":"2024-05-08T03:20:03.022099+0000","last_became_peered":"2024-05-08T03:20:03.022099+0000","last_unstale":"2024-05-08T03:20:16.096277+0000","last_undegraded":"2024-05-08T03:20:16.096277+0000","last_fullsized":"2024-05-08T03:20:16.096277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:59.767471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":40655336,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":4454,"num_read_kb":21153,"num_write":476,"num_write_kb":2798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":481376,"num_keys_recovered":30,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093623+0000","last_change":"2024-05-08T03:19:56.988049+0000","last_active":"2024-05-08T03:20:16.093623+0000","last_peered":"2024-05-08T03:20:16.093623+0000","last_clean":"2024-05-08T03:20:16.093623+0000","last_became_active":"2024-05-08T03:19:56.987737+0000","last_became_peered":"2024-05-08T03:19:56.987737+0000","last_unstale":"2024-05-08T03:20:16.093623+0000","last_undegraded":"2024-05-08T03:20:16.093623+0000","last_fullsized":"2024-05-08T03:20:16.093623+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:53.545130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"66'499","reported_seq":4288,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093742+0000","last_change":"2024-05-08T03:19:59.997039+0000","last_active":"2024-05-08T03:20:16.093742+0000","last_peered":"2024-05-08T03:20:16.093742+0000","last_clean":"2024-05-08T03:20:16.093742+0000","last_became_active":"2024-05-08T03:19:59.996378+0000","last_became_peered":"2024-05-08T03:19:59.996378+0000","last_unstale":"2024-05-08T03:20:16.093742+0000","last_undegraded":"2024-05-08T03:20:16.093742+0000","last_fullsized":"2024-05-08T03:20:16.093742+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:35:28.606332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":38667001,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":5486,"num_read_kb":38336,"num_write":729,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":6220729,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096082+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:20:16.096082+0000","last_peered":"2024-05-08T03:20:16.096082+0000","last_clean":"2024-05-08T03:20:16.096082+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:20:16.096082+0000","last_undegraded":"2024-05-08T03:20:16.096082+0000","last_fullsized":"2024-05-08T03:20:16.096082+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"67'282","reported_seq":1788,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.261480+0000","last_change":"2024-05-08T03:19:22.850234+0000","last_active":"2024-05-08T03:20:16.261480+0000","last_peered":"2024-05-08T03:20:16.261480+0000","last_clean":"2024-05-08T03:20:16.261480+0000","last_became_active":"2024-05-08T03:19:20.773661+0000","last_became_peered":"2024-05-08T03:19:20.773661+0000","last_unstale":"2024-05-08T03:20:16.261480+0000","last_undegraded":"2024-05-08T03:20:16.261480+0000","last_fullsized":"2024-05-08T03:20:16.261480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:44.776557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":10727456,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":3443,"num_read_kb":7380,"num_write":259,"num_write_kb":3170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"66'174","reported_seq":1293,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093720+0000","last_change":"2024-05-08T03:19:44.872224+0000","last_active":"2024-05-08T03:20:16.093720+0000","last_peered":"2024-05-08T03:20:16.093720+0000","last_clean":"2024-05-08T03:20:16.093720+0000","last_became_active":"2024-05-08T03:19:44.774602+0000","last_became_peered":"2024-05-08T03:19:44.774602+0000","last_unstale":"2024-05-08T03:20:16.093720+0000","last_undegraded":"2024-05-08T03:20:16.093720+0000","last_fullsized":"2024-05-08T03:20:16.093720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:50:19.719227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":77832661,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":992,"num_read_kb":12271,"num_write":204,"num_write_kb":15110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":5551754,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"66'181","reported_seq":603,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096439+0000","last_change":"2024-05-08T03:20:03.022267+0000","last_active":"2024-05-08T03:20:16.096439+0000","last_peered":"2024-05-08T03:20:16.096439+0000","last_clean":"2024-05-08T03:20:16.096439+0000","last_became_active":"2024-05-08T03:20:03.021701+0000","last_became_peered":"2024-05-08T03:20:03.021701+0000","last_unstale":"2024-05-08T03:20:16.096439+0000","last_undegraded":"2024-05-08T03:20:16.096439+0000","last_fullsized":"2024-05-08T03:20:16.096439+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:31:26.583348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":23773269,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":589,"num_read_kb":7315,"num_write":212,"num_write_kb":4258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":24576,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"67'144","reported_seq":1569,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093834+0000","last_change":"2024-05-08T03:19:51.297822+0000","last_active":"2024-05-08T03:20:16.093834+0000","last_peered":"2024-05-08T03:20:16.093834+0000","last_clean":"2024-05-08T03:20:16.093834+0000","last_became_active":"2024-05-08T03:19:50.937736+0000","last_became_peered":"2024-05-08T03:19:50.937736+0000","last_unstale":"2024-05-08T03:20:16.093834+0000","last_undegraded":"2024-05-08T03:20:16.093834+0000","last_fullsized":"2024-05-08T03:20:16.093834+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:52:56.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":43725413,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":1450,"num_read_kb":9853,"num_write":136,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2233399,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"67'377","reported_seq":1512,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.144957+0000","last_change":"2024-05-08T03:19:59.995470+0000","last_active":"2024-05-08T03:20:16.144957+0000","last_peered":"2024-05-08T03:20:16.144957+0000","last_clean":"2024-05-08T03:20:16.144957+0000","last_became_active":"2024-05-08T03:19:59.994538+0000","last_became_peered":"2024-05-08T03:19:59.994538+0000","last_unstale":"2024-05-08T03:20:16.144957+0000","last_undegraded":"2024-05-08T03:20:16.144957+0000","last_fullsized":"2024-05-08T03:20:16.144957+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:22:59.344920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":80155164,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":2900,"num_read_kb":12555,"num_write":548,"num_write_kb":16348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":8235378,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"66'259","reported_seq":1333,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093898+0000","last_change":"2024-05-08T03:19:39.181175+0000","last_active":"2024-05-08T03:20:16.093898+0000","last_peered":"2024-05-08T03:20:16.093898+0000","last_clean":"2024-05-08T03:20:16.093898+0000","last_became_active":"2024-05-08T03:19:39.090307+0000","last_became_peered":"2024-05-08T03:19:39.090307+0000","last_unstale":"2024-05-08T03:20:16.093898+0000","last_undegraded":"2024-05-08T03:20:16.093898+0000","last_fullsized":"2024-05-08T03:20:16.093898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:04.415751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":60238334,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":2729,"num_read_kb":38993,"num_write":322,"num_write_kb":38993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":993485,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"67'221","reported_seq":2237,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.181241+0000","last_change":"2024-05-08T03:19:59.997892+0000","last_active":"2024-05-08T03:20:16.181241+0000","last_peered":"2024-05-08T03:20:16.181241+0000","last_clean":"2024-05-08T03:20:16.181241+0000","last_became_active":"2024-05-08T03:19:59.997367+0000","last_became_peered":"2024-05-08T03:19:59.997367+0000","last_unstale":"2024-05-08T03:20:16.181241+0000","last_undegraded":"2024-05-08T03:20:16.181241+0000","last_fullsized":"2024-05-08T03:20:16.181241+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:26:00.262563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":55345474,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":1911,"num_read_kb":28364,"num_write":234,"num_write_kb":11966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":31431070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"66'261","reported_seq":1429,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.096547+0000","last_change":"2024-05-08T03:19:57.244251+0000","last_active":"2024-05-08T03:20:16.096547+0000","last_peered":"2024-05-08T03:20:16.096547+0000","last_clean":"2024-05-08T03:20:16.096547+0000","last_became_active":"2024-05-08T03:19:56.989657+0000","last_became_peered":"2024-05-08T03:19:56.989657+0000","last_unstale":"2024-05-08T03:20:16.096547+0000","last_undegraded":"2024-05-08T03:20:16.096547+0000","last_fullsized":"2024-05-08T03:20:16.096547+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:29:11.892125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":14566831,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":1329,"num_read_kb":6715,"num_write":303,"num_write_kb":10131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4608126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"67'121","reported_seq":858,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.227120+0000","last_change":"2024-05-08T03:20:03.024129+0000","last_active":"2024-05-08T03:20:16.227120+0000","last_peered":"2024-05-08T03:20:16.227120+0000","last_clean":"2024-05-08T03:20:16.227120+0000","last_became_active":"2024-05-08T03:20:03.023616+0000","last_became_peered":"2024-05-08T03:20:03.023616+0000","last_unstale":"2024-05-08T03:20:16.227120+0000","last_undegraded":"2024-05-08T03:20:16.227120+0000","last_fullsized":"2024-05-08T03:20:16.227120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:14.701992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":58070784,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":563,"num_read_kb":17869,"num_write":136,"num_write_kb":11280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":10514324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"66'265","reported_seq":2841,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093682+0000","last_change":"2024-05-08T03:20:03.024587+0000","last_active":"2024-05-08T03:20:16.093682+0000","last_peered":"2024-05-08T03:20:16.093682+0000","last_clean":"2024-05-08T03:20:16.093682+0000","last_became_active":"2024-05-08T03:20:03.023621+0000","last_became_peered":"2024-05-08T03:20:03.023621+0000","last_unstale":"2024-05-08T03:20:16.093682+0000","last_undegraded":"2024-05-08T03:20:16.093682+0000","last_fullsized":"2024-05-08T03:20:16.093682+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:22:25.668056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":54686297,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":3872,"num_read_kb":47784,"num_write":243,"num_write_kb":48666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":14483473,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"66'123","reported_seq":269,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093558+0000","last_change":"2024-05-08T03:20:06.097542+0000","last_active":"2024-05-08T03:20:16.093558+0000","last_peered":"2024-05-08T03:20:16.093558+0000","last_clean":"2024-05-08T03:20:16.093558+0000","last_became_active":"2024-05-08T03:20:06.033352+0000","last_became_peered":"2024-05-08T03:20:06.033352+0000","last_unstale":"2024-05-08T03:20:16.093558+0000","last_undegraded":"2024-05-08T03:20:16.093558+0000","last_fullsized":"2024-05-08T03:20:16.093558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:56:20.408783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":36171838,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":11,"num_read":1566,"num_read_kb":12594,"num_write":121,"num_write_kb":5279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"66'205","reported_seq":1190,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-08T03:20:16.093836+0000","last_change":"2024-05-08T03:19:47.769634+0000","last_active":"2024-05-08T03:20:16.093836+0000","last_peered":"2024-05-08T03:20:16.093836+0000","last_clean":"2024-05-08T03:20:16.093836+0000","last_became_active":"2024-05-08T03:19:47.769070+0000","last_became_peered":"2024-05-08T03:19:47.769070+0000","last_unstale":"2024-05-08T03:20:16.093836+0000","last_undegraded":"2024-05-08T03:20:16.093836+0000","last_fullsized":"2024-05-08T03:20:16.093836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:04.402464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":22091675,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":1217,"num_read_kb":27124,"num_write":271,"num_write_kb":20347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":13456776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1198158330,"num_objects":1188,"num_object_clones":229,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":418,"num_read":74371,"num_read_kb":608481,"num_write":9518,"num_write_kb":431226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1131,"num_bytes_recovered":160374736,"num_keys_recovered":229,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":755990528,"data_stored":1108243088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7734,"ondisk_log_size":7734,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542160,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":222616,"kb_used_data":181200,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94149224,"statfs":{"total":96636764160,"available":96408805376,"internally_reserved":0,"allocated":185548800,"data_stored":280671338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":60129542159,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":60129542161,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":226352,"kb_used_data":184908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94145488,"statfs":{"total":96636764160,"available":96404979712,"internally_reserved":0,"allocated":189345792,"data_stored":271313545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130012,"kb_used_data":94676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94241828,"statfs":{"total":96636764160,"available":96503631872,"internally_reserved":0,"allocated":96948224,"data_stored":146483427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":204040,"kb_used_data":176972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94167800,"statfs":{"total":96636764160,"available":96427827200,"internally_reserved":0,"allocated":181219328,"data_stored":258903166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542161,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":179810304,"data_stored":257929726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96137216,"data_stored":146097712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":188080128,"data_stored":270462317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":496740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":185786368,"data_stored":281318198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:20:21.923 INFO:tasks.thrashosds.thrasher:Setting 1.4 to [4, 2, 5, 6] 2024-05-08T03:20:21.923 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.4', '4', '2', '5', '6'] 2024-05-08T03:20:21.923 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.4 4 2 5 6 2024-05-08T03:20:21.925 INFO:teuthology.orchestra.run.smithi177.stdout:3858 read 0x3526b66 thru 0x3534861 (0xdcfc bytes) 2024-05-08T03:20:21.927 INFO:teuthology.orchestra.run.smithi177.stdout:3859 trunc from 0x48d3f8f to 0x30cb4b7 2024-05-08T03:20:21.955 INFO:teuthology.orchestra.run.smithi177.stdout:3860 trunc from 0x30cb4b7 to 0xc392f80 2024-05-08T03:20:21.955 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:21.953+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:21.960 INFO:teuthology.orchestra.run.smithi177.stdout:3861 write 0x84d549 thru 0x84e40f (0xec7 bytes) 2024-05-08T03:20:21.962 INFO:teuthology.orchestra.run.smithi177.stdout:3862 write 0x95ce35b thru 0x95d3f35 (0x5bdb bytes) 2024-05-08T03:20:21.965 INFO:teuthology.orchestra.run.smithi177.stdout:3863 write 0x369a051 thru 0x36a30a0 (0x9050 bytes) 2024-05-08T03:20:21.967 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:21.968 INFO:teuthology.orchestra.run.smithi177.stdout:3865 writesame 0xa045b8d thru 0xa04c19e data_size 0x27(0x6612 bytes) 2024-05-08T03:20:21.970 INFO:teuthology.orchestra.run.smithi177.stdout:3866 writesame 0x691d2d6 thru 0x6925607 data_size 0x7(0x8332 bytes) 2024-05-08T03:20:21.975 INFO:teuthology.orchestra.run.smithi177.stdout:3867 writesame 0x50fda69 thru 0x50ff149 data_size 0x1(0x16e1 bytes) 2024-05-08T03:20:21.981 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.981 INFO:teuthology.orchestra.run.smithi177.stdout:3869 writesame 0xba96260 thru 0xbaa34fb data_size 0x6(0xd29c bytes) 2024-05-08T03:20:21.989 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.989 INFO:teuthology.orchestra.run.smithi177.stdout:3871 read 0x8aebcb7 thru 0x8af8093 (0xc3dd bytes) 2024-05-08T03:20:21.989 INFO:teuthology.orchestra.run.smithi177.stdout:3872 punch from 0x248549d to 0x2488375, (0x2ed8 bytes) 2024-05-08T03:20:21.989 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:21.989 INFO:teuthology.orchestra.run.smithi177.stdout:3875 trunc from 0xc392f80 to 0xb9b87d3 2024-05-08T03:20:21.997 INFO:teuthology.orchestra.run.smithi177.stdout:3876 punch from 0x66b38bc to 0x66bde06, (0xa54a bytes) 2024-05-08T03:20:21.997 INFO:teuthology.orchestra.run.smithi177.stdout:3877 write 0x3c09d7d thru 0x3c11155 (0x73d9 bytes) 2024-05-08T03:20:21.999 INFO:teuthology.orchestra.run.smithi177.stdout:3880 punch from 0x8c1370c to 0x8c1ebad, (0xb4a1 bytes) 2024-05-08T03:20:21.999 INFO:teuthology.orchestra.run.smithi177.stdout:3881 read 0x7cd29fb thru 0x7cd413c (0x1742 bytes) 2024-05-08T03:20:21.999 INFO:teuthology.orchestra.run.smithi177.stdout:3882 write 0x31a9e0c thru 0x31ae5aa (0x479f bytes) 2024-05-08T03:20:22.001 INFO:teuthology.orchestra.run.smithi177.stdout:3885 punch from 0x39d0a67 to 0x39dee64, (0xe3fd bytes) 2024-05-08T03:20:22.001 INFO:teuthology.orchestra.run.smithi177.stdout:3886 write 0x409855d thru 0x4098b76 (0x61a bytes) 2024-05-08T03:20:22.005 INFO:teuthology.orchestra.run.smithi177.stdout:3887 read 0x3017a75 thru 0x302389b (0xbe27 bytes) 2024-05-08T03:20:22.005 INFO:teuthology.orchestra.run.smithi177.stdout:3888 write 0x459c16 thru 0x45a2d4 (0x6bf bytes) 2024-05-08T03:20:22.007 INFO:teuthology.orchestra.run.smithi177.stdout:3890 write 0x34586e3 thru 0x3464b34 (0xc452 bytes) 2024-05-08T03:20:22.009 INFO:teuthology.orchestra.run.smithi177.stdout:3891 write 0x9ad94c3 thru 0x9adc947 (0x3485 bytes) 2024-05-08T03:20:22.010 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:22.010 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:22.012 INFO:teuthology.orchestra.run.smithi177.stdout:3893 punch from 0x4aaf25f to 0x4ab3283, (0x4024 bytes) 2024-05-08T03:20:22.018 INFO:teuthology.orchestra.run.smithi177.stdout:3894 trunc from 0xb9b87d3 to 0x70168a2 2024-05-08T03:20:22.026 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:22.031 INFO:teuthology.orchestra.run.smithi177.stdout:3895 write 0x55ed9d4 thru 0x55f24e7 (0x4b14 bytes) 2024-05-08T03:20:22.033 INFO:teuthology.orchestra.run.smithi177.stdout:3896 writesame 0x511c91c thru 0x5121aa9 data_size 0x49(0x518e bytes) 2024-05-08T03:20:22.033 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:22.031+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:22.034 INFO:teuthology.orchestra.run.smithi177.stdout:3898 punch from 0x50ce9c9 to 0x50d14f9, (0x2b30 bytes) 2024-05-08T03:20:22.048 INFO:teuthology.orchestra.run.smithi177.stdout:3899 writesame 0xc7fe6f8 thru 0xc8013f7 data_size 0x5a(0x2d00 bytes) 2024-05-08T03:20:22.056 INFO:teuthology.orchestra.run.smithi177.stdout:3900 write 0xdfb2be1 thru 0xdfb7817 (0x4c37 bytes) 2024-05-08T03:20:22.059 INFO:teuthology.orchestra.run.smithi177.stdout:3902 clone 28 order 22 su 131072 sc 7 2024-05-08T03:20:22.116 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:20:22.117 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:22.118 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:22.133 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:22.131+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:22.154 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:22.154 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:22.170 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:22.170 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:22.182 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:22.188 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:22.189 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:22.208 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:22.208 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:22.232 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:22.232 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:22.234 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:22.232+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:22.235 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:22.273 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:22.273 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:22.310 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:22.310 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:22.315 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:22.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.223733300999999, 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:22.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:22.331 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.222012831000001, 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.332 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.336 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.22197939, 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:22.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:22.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.220553650999999, 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.341 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:22.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.220461037, 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.219950363000001, 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:22.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.219922855, 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.358 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.219516424, 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:22.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.362 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.219061046, 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:22.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:22.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:22.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.367 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.218532799, 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:22.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:22.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:22.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:22.371 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.218060301, 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:22.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217982169000001, 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.376 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:22.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:22.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217906747000001, 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.381 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:22.385 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217829584, 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.390 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217752061000001, 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217725149, 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.395 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:22.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217703939, 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:22.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217684158999999, 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.404 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.408 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217129633000001, 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.409 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:22.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.217110621, 2024-05-08T03:20:22.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:22.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.418 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:22.419 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:22.420 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-05-08T03:20:22.422 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:22.421+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:22.520 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:22.519+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:22.572 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:22.572 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:22.572 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:22.572 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:22.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:22.625 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:22.625+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:22.690 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:22.690 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:22.690 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:22.690 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:22.690 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:22.695 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:22.695 INFO:teuthology.orchestra.run.smithi177.stdout:3906 clone 29 order 20 su 32768 sc 15 2024-05-08T03:20:22.700 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:22.722 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:22.721+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:22.823 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:22.823+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:22.846 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:22.846 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:22.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:22.846 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:22.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 23.847796019, 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:22.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:22.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:22.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:22.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:22.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 23.847728555, 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.852 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:22.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:22.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:22.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:22.858 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 23.787894222999999, 2024-05-08T03:20:22.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.721386209999999, 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.865 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:22.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:22.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.694848564000001, 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.871 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:22.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:22.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.666936796000002, 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:22.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:22.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23281 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'92, mlcod=53'88)", 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.617391276999999, 2024-05-08T03:20:22.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074279843999999998, 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226784+0000", 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226785+0000", 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.299e-06 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226794+0000", 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5040000000000008e-06 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226810+0000", 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5781e-05 2024-05-08T03:20:22.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.226820+0000", 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9310000000000003e-06 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262249+0000", 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.035428501000000001 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.262269+0000", 2024-05-08T03:20:22.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0077000000000001e-05 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301028+0000", 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.038759184000000002 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.301064+0000", 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5567e-05 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:50996 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.173304+0000", 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6708707690000004, 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069887774, 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 50996 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173304+0000", 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173304+0000", 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173305+0000", 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.37e-07 2024-05-08T03:20:22.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173307+0000", 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0810000000000001e-06 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173313+0000", 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.348e-06 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173318+0000", 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:22.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.242419+0000", 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.06910057 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.242486+0000", 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7497999999999996e-05 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243192+0000", 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070565000000000005 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.889 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:50997 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6708502110000003, 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070252423999999994, 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 50997 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173325+0000", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8599999999999999e-07 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173327+0000", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1019999999999999e-06 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173334+0000", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6120000000000004e-06 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173339+0000", 2024-05-08T03:20:22.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2460000000000003e-06 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243218+0000", 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069878632999999996 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243278+0000", 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0443999999999999e-05 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243577+0000", 2024-05-08T03:20:22.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00029900100000000002 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:50998 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 344064~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.173345+0000", 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6708294910000001, 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070754912000000003, 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 50998 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173345+0000", 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.173345+0000", 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175140+0000", 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001794152 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175147+0000", 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2180000000000002e-06 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175158+0000", 2024-05-08T03:20:22.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1331999999999999e-05 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175165+0000", 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3390000000000004e-06 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243606+0000", 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.068440584999999998 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.243678+0000", 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2213999999999997e-05 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.244100+0000", 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00042207200000000001 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51000 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6689731300000004, 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070034812000000002, 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51000 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175202+0000", 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175205+0000", 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.43e-06 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175212+0000", 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1439999999999997e-06 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175217+0000", 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2989999999999996e-06 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.244508+0000", 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069290884999999997 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.244566+0000", 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7807999999999999e-05 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.245237+0000", 2024-05-08T03:20:22.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000670749 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51001 3.7 3:e2354cf1:::rbd_data.11829e0323ef.000000000000001d:1a [read 417792~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6689511320000001, 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071026390999999994, 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51001 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175224+0000", 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0200000000000003e-07 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175226+0000", 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1859999999999999e-06 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175233+0000", 2024-05-08T03:20:22.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4609999999999996e-06 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175238+0000", 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2379999999999997e-06 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.245267+0000", 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070028956000000003 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.245337+0000", 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0176e-05 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246250+0000", 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091297199999999996 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51002 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175244+0000", 2024-05-08T03:20:22.907 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6689305240000003, 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071040220000000001, 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51002 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175244+0000", 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175244+0000", 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175245+0000", 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4000000000000002e-07 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175247+0000", 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2110000000000001e-06 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175254+0000", 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4609999999999996e-06 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175259+0000", 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999998e-06 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246277+0000", 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071018681 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246285+0000", 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.357e-06 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51003 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:22.911 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6689101539999998, 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071047076000000001, 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51003 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:22.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175265+0000", 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5499999999999998e-07 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175267+0000", 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1509999999999998e-06 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175274+0000", 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.426e-06 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175279+0000", 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2120000000000002e-06 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246306+0000", 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071026813999999994 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246312+0000", 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0179999999999998e-06 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.915 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51004 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175285+0000", 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6688897449999995, 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071052522000000007, 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51004 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175285+0000", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175285+0000", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175286+0000", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.46e-07 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175288+0000", 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1299999999999999e-06 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175294+0000", 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.46e-06 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175299+0000", 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9749999999999998e-06 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246332+0000", 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071032895999999998 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246338+0000", 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7150000000000003e-06 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51005 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.919 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175305+0000", 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.668869816, 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071058287999999997, 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51005 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175305+0000", 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175305+0000", 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175306+0000", 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175308+0000", 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0470000000000001e-06 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175314+0000", 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3260000000000001e-06 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175319+0000", 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3159999999999996e-06 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246357+0000", 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071038089999999998 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246363+0000", 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0920000000000003e-06 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51006 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175326+0000", 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6688488159999997, 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071064635000000001, 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51006 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175326+0000", 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175326+0000", 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175327+0000", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9400000000000001e-07 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175329+0000", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2809999999999998e-06 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175335+0000", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4810000000000003e-06 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175340+0000", 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9169999999999997e-06 2024-05-08T03:20:22.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246384+0000", 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071043998999999997 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246391+0000", 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5629999999999997e-06 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.926 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51008 3.7 3.8f32ac47 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.175346+0000", 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6688286510000001, 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071071253000000001, 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51008 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175346+0000", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175346+0000", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175347+0000", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175349+0000", 2024-05-08T03:20:22.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.041e-06 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175355+0000", 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.175360+0000", 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1919999999999996e-06 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246411+0000", 2024-05-08T03:20:22.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071050786000000005 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246418+0000", 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6390000000000003e-06 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51012 3.8 3:1703486b:::rbd_header.1182fdc479eb:head [watch reconnect cookie 140222363529632 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.176104+0000", 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6680706299999999, 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074566035000000003, 2024-05-08T03:20:22.930 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51012 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176104+0000", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176104+0000", 2024-05-08T03:20:22.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176105+0000", 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6300000000000005e-07 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176108+0000", 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3529999999999999e-06 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176116+0000", 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1529999999999997e-06 2024-05-08T03:20:22.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176123+0000", 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9240000000000001e-06 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176170+0000", 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6412000000000002e-05 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176245+0000", 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4994999999999996e-05 2024-05-08T03:20:22.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.250670+0000", 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.074425535000000001 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:51014 3.7 3.8f32ac47 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:15.176959+0000", 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.6672162359999998, 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069488783999999998, 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:22.934 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 51014 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176959+0000", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176959+0000", 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176960+0000", 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5700000000000001e-07 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.176964+0000", 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1400000000000002e-06 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.177025+0000", 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1583999999999999e-05 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.177033+0000", 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1850000000000004e-06 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246440+0000", 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069406994 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:15.246447+0000", 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.024e-06 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.938 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:22.938 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:22.938 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:22.939 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:22.940 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:22.938+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:23.039 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:23.039+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:23.046 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.046 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:23.046 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:23.047 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.055 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:23.125 INFO:teuthology.orchestra.run.smithi177.stderr:set 1.4 pg_upmap_items mapping to [4->2,5->6] 2024-05-08T03:20:23.128 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:20:23.141 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:23.140+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:23.161 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.162 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:23.162 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:23.162 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:23.162 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.170 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:23.190 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone28 from 0xdfb7818 (overlap 0xdfb7818) to 0x0 2024-05-08T03:20:23.242 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:23.241+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:23.290 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.290 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:23.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.419561375000001, 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.291 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:20:23.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:20:23.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.419219196999997, 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:20:23.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:20:23.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:20:23.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:20:23.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.419180365000003, 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:20:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:20:23.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:20:23.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.419134331999999, 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:20:23.308 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:20:23.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:20:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.41811714, 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.314 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:20:23.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:20:23.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.318 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.418080889000002, 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:20:23.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:20:23.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:20:23.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.41805359, 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:20:23.325 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:23.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:20:23.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:20:23.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:20:23.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.418023896999998, 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.331 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:20:23.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.336 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.417998382, 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:20:23.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.407884215999999, 2024-05-08T03:20:23.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:23.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:20:23.345 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155545371000002, 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.348 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:20:23.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155527040000003, 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.352 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:20:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155502579, 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:20:23.357 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:20:23.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:20:23.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.361 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155476235999998, 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.362 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:20:23.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155458844999998, 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:20:23.366 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.370 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155441564, 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.373 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.374 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155414127, 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.375 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:20:23.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:20:23.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155396168000003, 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.379 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:23.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:23.383 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155379082000003, 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:23.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.386 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.387 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.155362097999998, 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.388 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.391 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.392 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.393 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-05-08T03:20:23.394 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:23.394 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:23.393+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:23.495 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:23.494+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:23.499 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.499 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:23.499 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:23.499 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:23.572 INFO:teuthology.orchestra.run.smithi177.stdout:3907 write 0x87fcb9d thru 0x880915c (0xc5c0 bytes) 2024-05-08T03:20:23.584 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.584 INFO:teuthology.orchestra.run.smithi177.stdout:3910 write 0x2ddab1f thru 0x2ddd89b (0x2d7d bytes) 2024-05-08T03:20:23.596 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:23.595+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:23.607 INFO:teuthology.orchestra.run.smithi177.stdout:3911 trunc from 0xdfb7818 to 0x1d4f111 2024-05-08T03:20:23.612 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.612 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:23.612 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:23.612 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:23.612 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.621 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:23.657 INFO:teuthology.orchestra.run.smithi177.stdout:3912 punch from 0xde9a64 to 0xdf58a3, (0xbe3f bytes) 2024-05-08T03:20:23.658 INFO:teuthology.orchestra.run.smithi177.stdout:3913 read 0x1c4698d thru 0x1c47b9c (0x1210 bytes) 2024-05-08T03:20:23.658 INFO:teuthology.orchestra.run.smithi177.stdout:3914 punch from 0xb55344 to 0xb60903, (0xb5bf bytes) 2024-05-08T03:20:23.658 INFO:teuthology.orchestra.run.smithi177.stdout:3915 writesame 0x5ab2ea thru 0x5b60d7 data_size 0x7b(0xadee bytes) 2024-05-08T03:20:23.668 INFO:teuthology.orchestra.run.smithi177.stdout:3916 read 0x3c094 thru 0x44fda (0x8f47 bytes) 2024-05-08T03:20:23.668 INFO:teuthology.orchestra.run.smithi177.stdout:3917 read 0x172530e thru 0x1726ef4 (0x1be7 bytes) 2024-05-08T03:20:23.677 INFO:teuthology.orchestra.run.smithi177.stdout:3921 writesame 0x46e84c4 thru 0x46f4dd0 data_size 0x1(0xc90d bytes) 2024-05-08T03:20:23.677 INFO:teuthology.orchestra.run.smithi177.stdout:3923 punch from 0x36bfd14 to 0x36ccd6c, (0xd058 bytes) 2024-05-08T03:20:23.689 INFO:teuthology.orchestra.run.smithi177.stdout:3925 write 0xadf4ee9 thru 0xae02f3e (0xe056 bytes) 2024-05-08T03:20:23.697 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:23.695+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:23.699 INFO:teuthology.orchestra.run.smithi177.stdout:3927 read 0x304eff thru 0x311327 (0xc429 bytes) 2024-05-08T03:20:23.702 INFO:teuthology.orchestra.run.smithi177.stdout:3929 writesame 0x3744d88 thru 0x3754799 data_size 0x1(0xfa12 bytes) 2024-05-08T03:20:23.728 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 47.519158345999998, 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.730 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:23.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:20:23.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599477829000001, 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.736 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:20:23.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599453758000003, 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.740 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:23.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.744 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599428035000003, 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:23.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599396317999997, 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:20:23.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.752 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599371214000001, 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:20:23.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:20:23.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:20:23.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599346408999999, 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:20:23.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:20:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599317865000003, 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:23.765 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599295343000001, 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:23.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:20:23.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.769 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599265002999999, 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:23.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.773 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.599243821999998, 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:20:23.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:23.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:20:23.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:20:23.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 32.786601886, 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.778 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:20:23.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:20:23.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:20:23.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:20:23.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.597927165000002, 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.784 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.575723822, 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.574027703999999, 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:20:23.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:20:23.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:20:23.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.554322916, 2024-05-08T03:20:23.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:20:23.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:20:23.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.554298956, 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:20:23.804 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:20:23.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:20:23.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:20:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:20:23.809 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.513707410999999, 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:20:23.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:20:23.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:20:23.813 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.500933514, 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 4.2401945799999998, 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:23.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:23.822 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:23.825 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:23.822+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3931 trunc from 0xae02f3f to 0xe562486 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3932 write 0x88ac21 thru 0x895da3 (0xb183 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3933 write 0xbeb4efc thru 0xbebd6a0 (0x87a5 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3934 punch from 0x606196f to 0x606d9d7, (0xc068 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3935 trunc from 0xe562486 to 0x310d9eb 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3938 write 0xd064e3c thru 0xd068cd1 (0x3e96 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3939 write 0x5cd663f thru 0x5cdbb32 (0x54f4 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3941 write 0x5453669 thru 0x546356e (0xff06 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:3943 read 0x3ddcbdc thru 0x3dde490 (0x18b5 bytes) 2024-05-08T03:20:23.826 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:3945 read 0x3492e48 thru 0x34a1101 (0xe2ba bytes) 2024-05-08T03:20:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:3946 trunc from 0xd068cd2 to 0xd84edf7 2024-05-08T03:20:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:3947 trunc from 0xd84edf7 to 0xc1627ff 2024-05-08T03:20:23.832 INFO:teuthology.orchestra.run.smithi177.stdout:3948 punch from 0x2c39313 to 0x2c3d8e0, (0x45cd bytes) 2024-05-08T03:20:23.832 INFO:teuthology.orchestra.run.smithi177.stdout:3949 punch from 0x5679751 to 0x567e293, (0x4b42 bytes) 2024-05-08T03:20:23.832 INFO:teuthology.orchestra.run.smithi177.stdout:3950 read 0x140797d thru 0x1416133 (0xe7b7 bytes) 2024-05-08T03:20:23.833 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.835 INFO:teuthology.orchestra.run.smithi177.stdout:3953 trunc from 0xc1627ff to 0xdd33be3 2024-05-08T03:20:23.849 INFO:teuthology.orchestra.run.smithi177.stdout:3955 trunc from 0xdd33be3 to 0x617eb22 2024-05-08T03:20:23.856 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.857 INFO:teuthology.orchestra.run.smithi177.stdout:3957 read 0x5a2152f thru 0x5a2b665 (0xa137 bytes) 2024-05-08T03:20:23.857 INFO:teuthology.orchestra.run.smithi177.stdout:3958 read 0x1efdccc thru 0x1f0695d (0x8c92 bytes) 2024-05-08T03:20:23.865 INFO:teuthology.orchestra.run.smithi177.stdout:3959 writesame 0x89eb563 thru 0x89efc90 data_size 0x3(0x472e bytes) 2024-05-08T03:20:23.865 INFO:teuthology.orchestra.run.smithi177.stdout:3960 read 0x104b5c2 thru 0x1055d38 (0xa777 bytes) 2024-05-08T03:20:23.865 INFO:teuthology.orchestra.run.smithi177.stdout:3961 write 0x745e2bf thru 0x7463918 (0x565a bytes) 2024-05-08T03:20:23.871 INFO:teuthology.orchestra.run.smithi177.stdout:3962 read 0xf984fc thru 0xfa6c88 (0xe78d bytes) 2024-05-08T03:20:23.879 INFO:teuthology.orchestra.run.smithi177.stdout:3964 writesame 0xa876783 thru 0xa884199 data_size 0x1(0xda17 bytes) 2024-05-08T03:20:23.879 INFO:teuthology.orchestra.run.smithi177.stdout:3966 write 0x129606e thru 0x1299afb (0x3a8e bytes) 2024-05-08T03:20:23.888 INFO:teuthology.orchestra.run.smithi177.stdout:3967 write 0xc996d94 thru 0xc9a35b1 (0xc81e bytes) 2024-05-08T03:20:23.898 INFO:teuthology.orchestra.run.smithi177.stdout:3970 read 0xac33c8b thru 0xac3c5c1 (0x8937 bytes) 2024-05-08T03:20:23.898 INFO:teuthology.orchestra.run.smithi177.stdout:3972 writesame 0x6955e76 thru 0x6964d5b data_size 0x3(0xeee6 bytes) 2024-05-08T03:20:23.909 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:23.909 INFO:teuthology.orchestra.run.smithi177.stdout:3974 write 0x88f6b48 thru 0x890414c (0xd605 bytes) 2024-05-08T03:20:23.919 INFO:teuthology.orchestra.run.smithi177.stdout:3975 write 0x563346c thru 0x563659a (0x312f bytes) 2024-05-08T03:20:23.920 INFO:teuthology.orchestra.run.smithi177.stdout:3976 writesame 0x5a723a thru 0x5b6663 data_size 0x1(0xf42a bytes) 2024-05-08T03:20:23.924 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:23.923+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:23.955 INFO:teuthology.orchestra.run.smithi177.stdout:3977 trunc from 0xc9a35b2 to 0x38b7104 2024-05-08T03:20:23.970 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:23.971 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:23.971 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:23.971 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:23.972 INFO:teuthology.orchestra.run.smithi177.stdout:3978 punch from 0x222aa95 to 0x22308f9, (0x5e64 bytes) 2024-05-08T03:20:23.974 INFO:teuthology.orchestra.run.smithi177.stdout:3979 trunc from 0x38b7104 to 0x23019cf 2024-05-08T03:20:23.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:23.999 INFO:teuthology.orchestra.run.smithi177.stdout:3980 writesame 0x79c3c43 thru 0x79cbdf1 data_size 0x1(0x81af bytes) 2024-05-08T03:20:23.999 INFO:teuthology.orchestra.run.smithi177.stdout:3982 write 0x25f013 thru 0x2660e1 (0x70cf bytes) 2024-05-08T03:20:24.004 INFO:teuthology.orchestra.run.smithi177.stdout:3983 trunc from 0x79cbdf2 to 0x6b20c1a 2024-05-08T03:20:24.021 INFO:teuthology.orchestra.run.smithi177.stdout:3986 write 0xbdbdc39 thru 0xbdc8f15 (0xb2dd bytes) 2024-05-08T03:20:24.025 INFO:teuthology.orchestra.run.smithi177.stdout:3987 write 0x15ffc thru 0x2461f (0xe624 bytes) 2024-05-08T03:20:24.027 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:24.027 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:24.027 INFO:teuthology.orchestra.run.smithi177.stdout:3995 write 0x97effec thru 0x97f0000 (0x15 bytes) 2024-05-08T03:20:24.031 INFO:teuthology.orchestra.run.smithi177.stdout:3996 write 0x8727006 thru 0x872f49e (0x8499 bytes) 2024-05-08T03:20:24.042 INFO:teuthology.orchestra.run.smithi177.stdout:3997 read 0x67ca42e thru 0x67ca79d (0x370 bytes) 2024-05-08T03:20:24.042 INFO:teuthology.orchestra.run.smithi177.stdout:3999 write 0xb452cfe thru 0xb45db8f (0xae92 bytes) 2024-05-08T03:20:24.046 INFO:teuthology.orchestra.run.smithi177.stdout:4001 read 0x2352586 thru 0x235842a (0x5ea5 bytes) 2024-05-08T03:20:24.054 INFO:teuthology.orchestra.run.smithi177.stdout:4002 write 0xe3c8b2d thru 0xe3ca226 (0x16fa bytes) 2024-05-08T03:20:24.058 INFO:teuthology.orchestra.run.smithi177.stdout:4003 read 0xba81d9d thru 0xba91261 (0xf4c5 bytes) 2024-05-08T03:20:24.058 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:24.058 INFO:teuthology.orchestra.run.smithi177.stdout:4006 write 0x92bb597 thru 0x92c60cf (0xab39 bytes) 2024-05-08T03:20:24.064 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:24.062+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:24.066 INFO:teuthology.orchestra.run.smithi177.stdout:4008 read 0xbd69a80 thru 0xbd6c68e (0x2c0f bytes) 2024-05-08T03:20:24.066 INFO:teuthology.orchestra.run.smithi177.stdout:4009 writesame 0x6954a20 thru 0x6963223 data_size 0x3e(0xe804 bytes) 2024-05-08T03:20:24.071 INFO:teuthology.orchestra.run.smithi177.stdout:4011 read 0x412b3b thru 0x4212bd (0xe783 bytes) 2024-05-08T03:20:24.071 INFO:teuthology.orchestra.run.smithi177.stdout:4012 read 0x28a619e thru 0x28b04c4 (0xa327 bytes) 2024-05-08T03:20:24.071 INFO:teuthology.orchestra.run.smithi177.stdout:4014 writesame 0x9c17f0b thru 0x9c1f5b0 data_size 0x1(0x76a6 bytes) 2024-05-08T03:20:24.088 INFO:teuthology.orchestra.run.smithi177.stdout:4015 write 0xdef4f56 thru 0xdefadb0 (0x5e5b bytes) 2024-05-08T03:20:24.091 INFO:teuthology.orchestra.run.smithi177.stdout:4016 writesame 0x39cf26a thru 0x39d144b data_size 0x1(0x21e2 bytes) 2024-05-08T03:20:24.114 INFO:teuthology.orchestra.run.smithi177.stdout:4017 trunc from 0xe3ca227 to 0x89f24d 2024-05-08T03:20:24.125 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:24.126 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:24.126 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:24.126 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:24.126 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:24.126 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:24.125+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:24.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:24.168 INFO:teuthology.orchestra.run.smithi177.stdout:4018 read 0x725810 thru 0x72ef27 (0x9718 bytes) 2024-05-08T03:20:24.168 INFO:teuthology.orchestra.run.smithi177.stdout:4019 read 0x785e6a thru 0x792a75 (0xcc0c bytes) 2024-05-08T03:20:24.179 INFO:teuthology.orchestra.run.smithi177.stdout:4020 write 0x503ea78 thru 0x50455e3 (0x6b6c bytes) 2024-05-08T03:20:24.186 INFO:teuthology.orchestra.run.smithi177.stdout:4023 clone 30 order 18 su 16384 sc 7 2024-05-08T03:20:24.227 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:24.225+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:24.282 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 52.406380820000003, 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.283 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:20:24.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:20:24.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064439628000002, 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.289 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:20:24.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064424228999997, 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.293 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064392587, 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.297 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:20:24.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064379254999999, 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.301 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:24.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:24.305 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064369460999998, 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.309 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064360047999998, 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:20:24.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:24.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.313 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064350527000002, 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.314 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:20:24.315 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:20:24.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:20:24.317 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064341186, 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.318 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.319 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.320 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:20:24.321 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064331971000001, 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:20:24.322 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.323 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.324 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.325 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.326 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064322609000001, 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.327 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.328 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.329 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.330 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064313177000002, 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.331 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.332 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.333 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:24.333 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:24.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.333 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.333 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:20:24.334 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:20:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064303760999998, 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:20:24.336 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.337 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.338 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:20:24.339 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.340 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064294742999998, 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:24.341 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.342 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.343 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.344 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064285699999999, 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.345 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.346 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.347 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.348 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064273407000002, 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.349 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.350 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.351 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.352 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064264139999999, 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.353 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:20:24.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:20:24.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064250282000003, 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.358 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.360 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:20:24.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.064240988999998, 2024-05-08T03:20:24.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.366 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 48.071226856000003, 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:24.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:20:24.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:20:24.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:20:24.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:20:24.371 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.371 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.371 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.371 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:24.371 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:24.371 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:24.371 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:24.373 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:24.371+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:24.373 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:24.474 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:24.472+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:24.482 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:24.483 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:24.483 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:24.483 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:24.492 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:24.497 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:24.497 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:24.518 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:24.518 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:24.532 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:24.532 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:24.549 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:24.549 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:24.566 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:24.568 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:24.568 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:24.575 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:24.574+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:24.584 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:24.584 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:24.599 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:24.599 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:24.602 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:24.602 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:24.605 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:24.605 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:24.609 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:24.675 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:24.673+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:24.754 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:24.754 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:24.762 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:24.807 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:24.806+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:24.867 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:24.867 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:24.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:24.918 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:24.917+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:24.979 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:24.979+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:25.022 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.022 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.031 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:25.080 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:25.079+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:25.145 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:20:25.189 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:25.189+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:25.190 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.190 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.198 INFO:teuthology.orchestra.run.smithi177.stdout:leaving image image_client.0-clone29 intact 2024-05-08T03:20:25.201 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:25.284 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-05-08T03:20:25.284 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:25.284+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:25.289 INFO:teuthology.orchestra.run.smithi177.stdout:4024 read 0x255d40 thru 0x258abe (0x2d7f bytes) 2024-05-08T03:20:25.290 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.308 INFO:teuthology.orchestra.run.smithi177.stdout:4026 writesame 0xaa02c0a thru 0xaa0390c data_size 0x1(0xd03 bytes) 2024-05-08T03:20:25.320 INFO:teuthology.orchestra.run.smithi177.stdout:4027 trunc from 0xaa0390d to 0x2caeb24 2024-05-08T03:20:25.342 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.342 INFO:teuthology.orchestra.run.smithi177.stdout:4030 punch from 0xe8decb to 0xe954d8, (0x760d bytes) 2024-05-08T03:20:25.360 INFO:teuthology.orchestra.run.smithi177.stdout:4031 writesame 0xc10d440 thru 0xc11302c data_size 0x1(0x5bed bytes) 2024-05-08T03:20:25.360 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.360 INFO:teuthology.orchestra.run.smithi177.stdout:4034 read 0x179b62b thru 0x17a8cbd (0xd693 bytes) 2024-05-08T03:20:25.360 INFO:teuthology.orchestra.run.smithi177.stdout:4035 writesame 0xbcd9867 thru 0xbce951b data_size 0x1(0xfcb5 bytes) 2024-05-08T03:20:25.366 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.366 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.377 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:25.386 INFO:teuthology.orchestra.run.smithi177.stdout:4037 write 0x89cc30b thru 0x89d4700 (0x83f6 bytes) 2024-05-08T03:20:25.387 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:25.386+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:25.396 INFO:teuthology.orchestra.run.smithi177.stdout:4038 punch from 0x8514646 to 0x8515f03, (0x18bd bytes) 2024-05-08T03:20:25.399 INFO:teuthology.orchestra.run.smithi177.stdout:4040 trunc from 0xc11302d to 0xe08ed72 2024-05-08T03:20:25.404 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.404 INFO:teuthology.orchestra.run.smithi177.stdout:4042 read 0x943d155 thru 0x944828d (0xb139 bytes) 2024-05-08T03:20:25.404 INFO:teuthology.orchestra.run.smithi177.stdout:4043 punch from 0x18e3cb7 to 0x18e6cf3, (0x303c bytes) 2024-05-08T03:20:25.404 INFO:teuthology.orchestra.run.smithi177.stdout:4044 write 0x69169ab thru 0x691b85c (0x4eb2 bytes) 2024-05-08T03:20:25.414 INFO:teuthology.orchestra.run.smithi177.stdout:4045 trunc from 0xe08ed72 to 0xcb727cb 2024-05-08T03:20:25.420 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.420 INFO:teuthology.orchestra.run.smithi177.stdout:4048 writesame 0xc79eff thru 0xc7b006 data_size 0x14(0x1108 bytes) 2024-05-08T03:20:25.425 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.425 INFO:teuthology.orchestra.run.smithi177.stdout:4052 punch from 0x640a426 to 0x640e7ca, (0x43a4 bytes) 2024-05-08T03:20:25.425 INFO:teuthology.orchestra.run.smithi177.stdout:4053 write 0xc59f1c8 thru 0xc5a1574 (0x23ad bytes) 2024-05-08T03:20:25.431 INFO:teuthology.orchestra.run.smithi177.stdout:4054 read 0x156f63a thru 0x157b64f (0xc016 bytes) 2024-05-08T03:20:25.431 INFO:teuthology.orchestra.run.smithi177.stdout:4055 write 0x88686b9 thru 0x8868fc2 (0x90a bytes) 2024-05-08T03:20:25.433 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.445 INFO:teuthology.orchestra.run.smithi177.stdout:4059 trunc from 0xcb727cb to 0x2e3cc2a 2024-05-08T03:20:25.459 INFO:teuthology.orchestra.run.smithi177.stdout:4060 read 0x5bf5af thru 0x5ced43 (0xf795 bytes) 2024-05-08T03:20:25.469 INFO:teuthology.orchestra.run.smithi177.stdout:4061 trunc from 0x2e3cc2a to 0xaaab0c5 2024-05-08T03:20:25.476 INFO:teuthology.orchestra.run.smithi177.stdout:4062 read 0x3d59fa7 thru 0x3d5e3d1 (0x442b bytes) 2024-05-08T03:20:25.476 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.476 INFO:teuthology.orchestra.run.smithi177.stdout:4064 read 0x4ab6709 thru 0x4ab8126 (0x1a1e bytes) 2024-05-08T03:20:25.476 INFO:teuthology.orchestra.run.smithi177.stdout:4066 read 0x59d055e thru 0x59d6fdf (0x6a82 bytes) 2024-05-08T03:20:25.477 INFO:teuthology.orchestra.run.smithi177.stdout:4067 read 0x92b5dac thru 0x92c1e68 (0xc0bd bytes) 2024-05-08T03:20:25.477 INFO:teuthology.orchestra.run.smithi177.stdout:4068 write 0x23619f5 thru 0x2368f06 (0x7512 bytes) 2024-05-08T03:20:25.484 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:25.482+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:25.485 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.492 INFO:teuthology.orchestra.run.smithi177.stdout:4073 write 0xc3bdb56 thru 0xc3c102b (0x34d6 bytes) 2024-05-08T03:20:25.498 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.505 INFO:teuthology.orchestra.run.smithi177.stdout:4075 write 0xd4c7355 thru 0xd4cdfee (0x6c9a bytes) 2024-05-08T03:20:25.513 INFO:teuthology.orchestra.run.smithi177.stdout:4076 write 0xb76b472 thru 0xb7772d9 (0xbe68 bytes) 2024-05-08T03:20:25.517 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.517 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.519 INFO:teuthology.orchestra.run.smithi177.stdout:4077 flatten 2024-05-08T03:20:25.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:25.617 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:25.616+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:25.624 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.639 INFO:teuthology.orchestra.run.smithi177.stdout:4079 trunc from 0xd4cdfef to 0xab652a 2024-05-08T03:20:25.655 INFO:teuthology.orchestra.run.smithi177.stdout:4080 read 0xfc3b7 thru 0x10937c (0xcfc6 bytes) 2024-05-08T03:20:25.661 INFO:teuthology.orchestra.run.smithi177.stdout:4081 trunc from 0xab652a to 0x49afabf 2024-05-08T03:20:25.666 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:25.666 INFO:teuthology.orchestra.run.smithi177.stdout:4083 clone 31 order 20 su 4096 sc 13 2024-05-08T03:20:25.685 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:25.683+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:25.703 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.703 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.712 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:25.786 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:25.784+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:25.857 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.857 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.866 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:25.910 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:25.909+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:25.970 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:25.970 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:25.979 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:26.020 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:26.019+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:26.089 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:26.087+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:26.125 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:26.125 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:26.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:26.139 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:20:26.195 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:26.194+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:26.246 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:26.247 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:26.255 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:26.261 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-05-08T03:20:26.299 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:26.297+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:26.354 INFO:teuthology.orchestra.run.smithi177.stdout:4084 write 0x3af34d4 thru 0x3b034aa (0xffd7 bytes) 2024-05-08T03:20:26.360 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:26.360 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:26.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:26.389 INFO:teuthology.orchestra.run.smithi177.stdout:4085 write 0x9aa1b61 thru 0x9aaf82d (0xdccd bytes) 2024-05-08T03:20:26.409 INFO:teuthology.orchestra.run.smithi177.stdout:4087 punch from 0x713d1ca to 0x7143926, (0x675c bytes) 2024-05-08T03:20:26.409 INFO:teuthology.orchestra.run.smithi177.stdout:4089 read 0x1119a53 thru 0x1128b8e (0xf13c bytes) 2024-05-08T03:20:26.409 INFO:teuthology.orchestra.run.smithi177.stdout:4091 punch from 0x5db2b2f to 0x5dc00b4, (0xd585 bytes) 2024-05-08T03:20:26.409 INFO:teuthology.orchestra.run.smithi177.stdout:4092 write 0x43f670a thru 0x43fd523 (0x6e1a bytes) 2024-05-08T03:20:26.412 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:26.410+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:26.430 INFO:teuthology.orchestra.run.smithi177.stdout:4093 write 0x9d4054b thru 0x9d4b93b (0xb3f1 bytes) 2024-05-08T03:20:26.444 INFO:teuthology.orchestra.run.smithi177.stdout:4094 write 0x90d87ab thru 0x90db373 (0x2bc9 bytes) 2024-05-08T03:20:26.446 INFO:teuthology.orchestra.run.smithi177.stdout:4096 read 0x49f0774 thru 0x49f920a (0x8a97 bytes) 2024-05-08T03:20:26.446 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:26.446 INFO:teuthology.orchestra.run.smithi177.stdout:4099 read 0x4ac7c90 thru 0x4aca7c2 (0x2b33 bytes) 2024-05-08T03:20:26.457 INFO:teuthology.orchestra.run.smithi177.stdout:4100 write 0xea5a8ed thru 0xea5d23a (0x294e bytes) 2024-05-08T03:20:26.463 INFO:teuthology.orchestra.run.smithi177.stdout:4101 writesame 0xc277785 thru 0xc278fef data_size 0x13(0x186b bytes) 2024-05-08T03:20:26.466 INFO:teuthology.orchestra.run.smithi177.stdout:4102 write 0x55566c5 thru 0x5559a62 (0x339e bytes) 2024-05-08T03:20:26.473 INFO:teuthology.orchestra.run.smithi177.stdout:4103 read 0xa6c2f15 thru 0xa6c800a (0x50f6 bytes) 2024-05-08T03:20:26.474 INFO:teuthology.orchestra.run.smithi177.stdout:4104 write 0xe91aa2d thru 0xe927134 (0xc708 bytes) 2024-05-08T03:20:26.490 INFO:teuthology.orchestra.run.smithi177.stdout:4105 read 0xa6b7d01 thru 0xa6c2094 (0xa394 bytes) 2024-05-08T03:20:26.490 INFO:teuthology.orchestra.run.smithi177.stdout:4107 write 0x1474574 thru 0x1476e38 (0x28c5 bytes) 2024-05-08T03:20:26.492 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:26.491+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:26.495 INFO:teuthology.orchestra.run.smithi177.stdout:4108 read 0x540ed56 thru 0x5418e72 (0xa11d bytes) 2024-05-08T03:20:26.496 INFO:teuthology.orchestra.run.smithi177.stdout:4109 punch from 0xd47974 to 0xd55477, (0xdb03 bytes) 2024-05-08T03:20:26.496 INFO:teuthology.orchestra.run.smithi177.stdout:4110 clone 32 order 22 su 16384 sc 15 2024-05-08T03:20:26.519 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:26.519 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:26.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:26.593 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:26.592+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:26.606 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:26.634 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:26.634 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:26.642 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:26.694 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:26.692+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:26.727 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:26.727 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:26.744 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:26.744 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:26.747 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:26.747 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:26.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:26.761 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:26.761 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:26.777 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:26.777 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:26.794 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:26.795 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:26.795 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:26.793+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:26.796 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:26.812 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:26.812 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:26.829 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:26.830 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:26.833 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:26.833 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:26.864 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:26.864 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:26.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:26.864 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:26.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.864 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 50.653680027999997, 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:20:26.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:20:26.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:20:26.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733999511, 2024-05-08T03:20:26.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:20:26.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:20:26.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733975440000002, 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:26.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.876 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.877 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:20:26.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.879 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733949717000002, 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733918000000003, 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.884 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:20:26.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:20:26.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733892896, 2024-05-08T03:20:26.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:20:26.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:20:26.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733868090999998, 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:20:26.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:20:26.896 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733839547000002, 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.897 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733817025, 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.901 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:20:26.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733786684999998, 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.905 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.733765503999997, 2024-05-08T03:20:26.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.913 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 35.921123567999999, 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.914 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:20:26.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:20:26.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:26.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:26.920 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.732448847000001, 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:20:26.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:20:26.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:20:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.925 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.710245504, 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:26.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:20:26.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:20:26.929 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.708549386000001, 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.688844597999999, 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.934 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.688820637999999, 2024-05-08T03:20:26.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:20:26.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:20:26.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:26.944 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.648229093000001, 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.635455195999999, 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:20:26.948 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:20:26.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:20:26.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 7.3747162619999997, 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:26.952 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:26.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:26.956 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:26.958 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:26.958 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:26.956+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:27.058 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:27.056+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:27.103 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:27.104 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:27.104 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:27.104 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:27.112 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:27.195 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone31 from 0xea5d23b (overlap 0x49afabf) to 0x0 2024-05-08T03:20:27.196 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:27.194+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:27.259 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:27.260 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:27.260 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:27.260 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:27.260 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:27.261 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:27.258+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:27.269 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:27.361 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:27.359+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:27.422 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:27.422 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:27.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:27.422 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.545884694000002, 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.423 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:20:27.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:20:27.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:20:27.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:27.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:20:27.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203943502000001, 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:20:27.429 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:20:27.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:20:27.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.433 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203928103000003, 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203896460999999, 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.438 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:27.442 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203883128999998, 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:27.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:20:27.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:20:27.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203873334999997, 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.447 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:20:27.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.451 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203863921999996, 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:27.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:20:27.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203854401000001, 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.459 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203845059999999, 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.460 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:27.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:27.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:20:27.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:20:27.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:20:27.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:20:27.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:20:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:20:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203835845, 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.469 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.470 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.470 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.470 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.470 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:20:27.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.470 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:27.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.472 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:20:27.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:20:27.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:20:27.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:20:27.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:27.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:20:27.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.477 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.478 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.478 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.478 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:27.478 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203826483, 2024-05-08T03:20:27.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:20:27.479 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.479 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.479 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.479 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.479 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.480 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.480 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:20:27.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.480 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.481 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:27.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.481 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.481 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:27.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:20:27.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:20:27.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:20:27.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:20:27.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:20:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:20:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:20:27.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:20:27.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:20:27.487 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.487 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.488 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.488 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.488 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:27.488 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203817051000001, 2024-05-08T03:20:27.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:20:27.489 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.489 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.489 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.489 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.489 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.490 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.490 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:20:27.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.490 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:27.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:20:27.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:27.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:20:27.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:20:27.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.495 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.495 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:20:27.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:20:27.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:20:27.497 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.497 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.497 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.498 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.498 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:27.498 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203807634999997, 2024-05-08T03:20:27.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.499 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:20:27.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.500 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:27.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:27.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:27.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:20:27.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:20:27.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:20:27.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:20:27.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:20:27.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:20:27.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:20:27.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:20:27.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.507 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203798616999997, 2024-05-08T03:20:27.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:20:27.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.510 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:27.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:27.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:20:27.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:20:27.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:20:27.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:20:27.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:20:27.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:20:27.516 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.516 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203789573999998, 2024-05-08T03:20:27.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.518 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:20:27.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:20:27.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:20:27.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.523 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203777281000001, 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.524 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:27.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203768013999998, 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.528 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203754156000002, 2024-05-08T03:20:27.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:27.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:20:27.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:20:27.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.537 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.203744862999997, 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:20:27.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:20:27.541 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 51.210730730000002, 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:27.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:27.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:20:27.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:20:27.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:27.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:20:27.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:20:27.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:20:27.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:20:27.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.549 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:27.549 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:27.549 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:27.550 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-05-08T03:20:27.551 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:27.552 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:27.549+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:27.610 INFO:teuthology.orchestra.run.smithi177.stdout:4111 writesame 0xfa27dc thru 0xfa2eaa data_size 0x7(0x6cf bytes) 2024-05-08T03:20:27.652 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:27.650+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:27.658 INFO:teuthology.orchestra.run.smithi177.stdout:4113 write 0x41d4225 thru 0x41db2c9 (0x70a5 bytes) 2024-05-08T03:20:27.687 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:27.688 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:27.688 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:27.688 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:27.699 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:27.699 INFO:teuthology.orchestra.run.smithi177.stdout:4115 clone 33 order 25 su 4194304 sc 5 2024-05-08T03:20:27.753 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:27.752+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:27.806 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:27.806 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:27.813 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:27.855 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:27.853+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:27.955 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:27.953+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:27.959 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:27.959 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:27.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:27.959 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:27.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.959 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.960 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:27.960 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.757591523999999, 2024-05-08T03:20:27.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:27.960 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.960 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.960 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:27.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:27.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.964 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.757582073999998, 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:27.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.757569338000003, 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:27.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:27.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.757558762000002, 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:27.973 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:27.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:27.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:27.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:27.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:27.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:27.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:27.980 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.757549374999996, 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.982 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:27.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.982 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:27.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:27.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:27.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:27.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:27.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:27.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:27.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:27.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.989 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:27.990 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.757540194999997, 2024-05-08T03:20:27.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:27.990 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.990 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.990 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:27.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:27.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:27.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:27.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:27.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:27.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:27.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:27.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:27.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:27.998 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526715160000002, 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:27.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:28.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:28.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:28.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526682923000003, 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:28.004 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:28.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526669593000001, 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.015 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526654618999999, 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526643309999997, 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.025 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:28.026 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526625641000003, 2024-05-08T03:20:28.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:28.026 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.026 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.026 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:28.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:28.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:28.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:28.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:28.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:28.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:28.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.033 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.034 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.034 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.034 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:28.034 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526610443000003, 2024-05-08T03:20:28.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:28.035 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.035 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.035 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.035 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.035 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:28.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:28.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:28.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:28.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:28.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:28.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:28.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:28.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:28.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.526596349000002, 2024-05-08T03:20:28.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.045 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:28.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:28.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:28.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:28.050 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.751642376, 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:28.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:28.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.055 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.828748574000002, 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.828735086000002, 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.828706105000002, 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:28.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:28.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:28.067 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.828679499000003, 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:28.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:28.071 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.395931238000003, 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:28.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.076 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.077 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.077 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:28.078 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:28.079 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:28.076+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:28.144 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6] out_osds: [4, 7, 0, 5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:28.144 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:28.144 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-05-08T03:20:28.144 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T03:20:28.144 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T03:20:28.144 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-05-08T03:20:28.155 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:20:28.205 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:28.204+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:28.228 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone32 from 0xea5d23b (overlap 0xea5d23b) to 0x2edf6d8 2024-05-08T03:20:28.256 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:28.256 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70843 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245708+0000", 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0088428710000000004, 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0088693969999999994, 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70843 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.257 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245708+0000", 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245708+0000", 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245708+0000", 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.39e-07 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245710+0000", 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2670000000000001e-06 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245717+0000", 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3690000000000003e-06 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245722+0000", 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2159999999999997e-06 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70844 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245728+0000", 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0088223409999999992, 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089148860000000003, 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.260 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70844 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245728+0000", 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245728+0000", 2024-05-08T03:20:28.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245729+0000", 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7899999999999999e-07 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245731+0000", 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1780000000000002e-06 2024-05-08T03:20:28.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245738+0000", 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8229999999999997e-06 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245743+0000", 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.263 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70845 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245751+0000", 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0087998469999999995, 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089501540000000001, 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70845 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245751+0000", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245751+0000", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245751+0000", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7000000000000005e-07 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245754+0000", 2024-05-08T03:20:28.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1459999999999999e-06 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245760+0000", 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1650000000000003e-06 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245765+0000", 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1939999999999997e-06 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70846 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245772+0000", 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0087789640000000002, 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089842689999999996, 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70846 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245772+0000", 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245772+0000", 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245772+0000", 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2200000000000004e-07 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245774+0000", 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1339999999999998e-06 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245781+0000", 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5629999999999997e-06 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245786+0000", 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0159999999999999e-06 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.269 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70847 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245792+0000", 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0087583039999999997, 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0090192650000000003, 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.270 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70847 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245792+0000", 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245792+0000", 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245793+0000", 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3700000000000001e-07 2024-05-08T03:20:28.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245799+0000", 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7339999999999996e-06 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245805+0000", 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9600000000000003e-06 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245812+0000", 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0580000000000003e-06 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70817 3.1c 3:38b47a5d:::rbd_data.11828f9dff47.000000000000003f:22 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245278+0000", 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0092728459999999995, 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0095963739999999995, 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70817 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245278+0000", 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245278+0000", 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245278+0000", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5299999999999999e-07 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245280+0000", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.17e-06 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245287+0000", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5300000000000002e-06 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245292+0000", 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1909999999999999e-06 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.250998+0000", 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0057063230000000001 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.251076+0000", 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7626000000000005e-05 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70911 3.0 3.b0dd1240 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.246018+0000", 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.008532698, 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0089235459999999992, 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70911 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.246018+0000", 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.246018+0000", 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.246018+0000", 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6800000000000001e-07 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.246021+0000", 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3479999999999998e-06 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.246028+0000", 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8240000000000002e-06 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.246033+0000", 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1880000000000001e-06 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70834 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245511+0000", 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0090393769999999995, 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0094861630000000006, 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70834 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.280 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245511+0000", 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245511+0000", 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245512+0000", 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:28.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245514+0000", 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.075e-06 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245520+0000", 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1e-06 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245525+0000", 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2050000000000001e-06 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70835 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245532+0000", 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0090189610000000007, 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0095206539999999999, 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.283 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70835 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245532+0000", 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245532+0000", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245532+0000", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0500000000000004e-07 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245534+0000", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.154e-06 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245541+0000", 2024-05-08T03:20:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9319999999999999e-06 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245547+0000", 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2170000000000002e-06 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70836 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245556+0000", 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0089943460000000003, 2024-05-08T03:20:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0095583999999999999, 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70836 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245556+0000", 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245556+0000", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245557+0000", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.27e-07 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245559+0000", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245566+0000", 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9269999999999999e-06 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245572+0000", 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4990000000000002e-06 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70837 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245579+0000", 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0089721479999999992, 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0095913270000000002, 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70837 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245579+0000", 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245579+0000", 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245579+0000", 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.96e-07 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245581+0000", 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1849999999999998e-06 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245587+0000", 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2380000000000002e-06 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245595+0000", 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3119999999999996e-06 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70838 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245602+0000", 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0089491280000000006, 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0096235650000000006, 2024-05-08T03:20:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70838 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245602+0000", 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245602+0000", 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245602+0000", 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245604+0000", 2024-05-08T03:20:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.249e-06 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245611+0000", 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5119999999999997e-06 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245616+0000", 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.643e-06 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70839 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245623+0000", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0089277590000000004, 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.009656698, 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70839 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245623+0000", 2024-05-08T03:20:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245623+0000", 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245623+0000", 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7399999999999999e-07 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245625+0000", 2024-05-08T03:20:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0660000000000002e-06 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245632+0000", 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0779999999999998e-06 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245638+0000", 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6289999999999998e-06 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70840 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245644+0000", 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0089064880000000006, 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0096831390000000003, 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70840 2024-05-08T03:20:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245644+0000", 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245644+0000", 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245645+0000", 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:20:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245647+0000", 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4430000000000002e-06 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245653+0000", 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4570000000000002e-06 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245659+0000", 2024-05-08T03:20:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3530000000000002e-06 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70841 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245666+0000", 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0088851020000000006, 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097137249999999994, 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70841 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245666+0000", 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245666+0000", 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245666+0000", 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245668+0000", 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1119999999999999e-06 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245675+0000", 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7870000000000004e-06 2024-05-08T03:20:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245680+0000", 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3990000000000003e-06 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70842 3.4 3.56a6d7a4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.245687+0000", 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0088638850000000002, 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0097473170000000001, 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70842 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245687+0000", 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245687+0000", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245687+0000", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7199999999999999e-07 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245690+0000", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.187e-06 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245696+0000", 2024-05-08T03:20:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2659999999999998e-06 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.245701+0000", 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2580000000000003e-06 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 16 2024-05-08T03:20:28.308 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:28.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:28.310 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:28.308+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:28.318 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T03:20:28.318 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:28.412 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:28.409+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:28.512 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:28.510+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:28.519 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:28.519 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:28.519 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:28.520 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:28.520 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:28.535 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:28.613 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:28.611+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:28.713 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:28.712+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:28.760 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.971777248999999, 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.761 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 38.8245431, 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.767 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:28.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:28.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 28.634402933000001, 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:28.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:28.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:28.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:28.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 28.609662387, 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:28.779 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:28.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67862 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237320+0000", 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5197505529999999, 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069307490999999999, 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67862 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237320+0000", 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237320+0000", 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237321+0000", 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7599999999999997e-07 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237323+0000", 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.058e-06 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237329+0000", 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9869999999999996e-06 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237334+0000", 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.865e-06 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306294+0000", 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.068960635000000006 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306363+0000", 2024-05-08T03:20:28.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8332999999999998e-05 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306628+0000", 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00026503699999999998 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67863 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5197298990000001, 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069622329999999996, 2024-05-08T03:20:28.789 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67863 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-07 2024-05-08T03:20:28.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237344+0000", 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237349+0000", 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7339999999999996e-06 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237354+0000", 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.605e-06 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306659+0000", 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069304893000000006 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306767+0000", 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000108385 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306963+0000", 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00019608200000000001 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67864 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5197109229999999, 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069780320000000007, 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67864 2024-05-08T03:20:28.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:28.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237362+0000", 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0679999999999999e-06 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237369+0000", 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4269999999999996e-06 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237373+0000", 2024-05-08T03:20:28.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.296e-06 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306990+0000", 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069617100000000001 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307041+0000", 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0796e-05 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307140+0000", 2024-05-08T03:20:28.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9189000000000002e-05 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67865 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5196923180000002, 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070114813999999998, 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.799 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67865 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:28.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.96e-07 2024-05-08T03:20:28.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237381+0000", 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237386+0000", 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.48e-06 2024-05-08T03:20:28.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237392+0000", 2024-05-08T03:20:28.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5620000000000003e-06 2024-05-08T03:20:28.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307171+0000", 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069779439999999998 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307237+0000", 2024-05-08T03:20:28.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5838000000000002e-05 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307493+0000", 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025623099999999999 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.806 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67866 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5196730189999998, 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070792377000000004, 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67866 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999999e-07 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237401+0000", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7630000000000001e-06 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237407+0000", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7169999999999996e-06 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237412+0000", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4729999999999999e-06 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307524+0000", 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070111455000000003 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307595+0000", 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1178000000000002e-05 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308190+0000", 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000595431 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.810 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67867 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5196526299999999, 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072514764999999995, 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.811 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67867 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237419+0000", 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7899999999999999e-07 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237421+0000", 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9860000000000001e-06 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237426+0000", 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.614e-06 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237431+0000", 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2680000000000002e-06 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308233+0000", 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070801136000000001 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308945+0000", 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00071273699999999996 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309933+0000", 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0009876450000000001 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67868 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.815 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.519633461, 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072516667000000007, 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67868 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239091+0000", 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001654006 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239096+0000", 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.921e-06 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239107+0000", 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0781000000000001e-05 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239114+0000", 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.939e-06 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309950+0000", 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070835871999999994 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309954+0000", 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1479999999999999e-06 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67869 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.819 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5179438780000001, 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070844969999999993, 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67869 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:28.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239128+0000", 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239130+0000", 2024-05-08T03:20:28.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.655e-06 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239138+0000", 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2690000000000002e-06 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239143+0000", 2024-05-08T03:20:28.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6690000000000003e-06 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309966+0000", 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070823102999999998 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309972+0000", 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6350000000000003e-06 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67870 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5179201500000001, 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070850033000000007, 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67870 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239153+0000", 2024-05-08T03:20:28.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2189999999999998e-06 2024-05-08T03:20:28.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239160+0000", 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7170000000000002e-06 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239166+0000", 2024-05-08T03:20:28.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4140000000000002e-06 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309994+0000", 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070828156000000003 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310001+0000", 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0310000000000001e-06 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67871 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.517898229, 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070908609999999997, 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.830 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.831 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.831 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.831 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.831 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.831 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67871 2024-05-08T03:20:28.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:28.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239175+0000", 2024-05-08T03:20:28.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239182+0000", 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5640000000000002e-06 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239187+0000", 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9409999999999998e-06 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310071+0000", 2024-05-08T03:20:28.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070883710000000003 2024-05-08T03:20:28.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310081+0000", 2024-05-08T03:20:28.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0716999999999999e-05 2024-05-08T03:20:28.838 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.838 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.838 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.838 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67876 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.517787784, 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070745004, 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.839 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.840 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67876 2024-05-08T03:20:28.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.840 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:28.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:28.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:28.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:20:28.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239286+0000", 2024-05-08T03:20:28.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0329999999999998e-06 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239291+0000", 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.57e-06 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239296+0000", 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5789999999999997e-06 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310022+0000", 2024-05-08T03:20:28.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070725882000000004 2024-05-08T03:20:28.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310028+0000", 2024-05-08T03:20:28.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:28.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.847 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.847 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67877 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.848 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:28.848 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.5177689590000001, 2024-05-08T03:20:28.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070752494999999999, 2024-05-08T03:20:28.848 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.848 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.848 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67877 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:28.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:28.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239305+0000", 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:28.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.853 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.853 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239311+0000", 2024-05-08T03:20:28.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:20:28.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239316+0000", 2024-05-08T03:20:28.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.527e-06 2024-05-08T03:20:28.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310048+0000", 2024-05-08T03:20:28.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070732489999999995 2024-05-08T03:20:28.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310054+0000", 2024-05-08T03:20:28.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2269999999999998e-06 2024-05-08T03:20:28.856 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.856 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.856 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67878 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.517749091, 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070795459000000005, 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.857 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67878 2024-05-08T03:20:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:28.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:28.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8799999999999998e-07 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239324+0000", 2024-05-08T03:20:28.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239330+0000", 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8050000000000003e-06 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239335+0000", 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5029999999999999e-06 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310110+0000", 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070775530000000003 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310117+0000", 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2089999999999999e-06 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67879 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.517730579, 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070805241000000005, 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67879 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239341+0000", 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239343+0000", 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9400000000000001e-06 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239349+0000", 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.533e-06 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239354+0000", 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4549999999999997e-06 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310138+0000", 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070784480999999996 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310146+0000", 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67880 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:28.867 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.517481219, 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070583059000000004, 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67880 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2200000000000003e-07 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239593+0000", 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8059999999999999e-06 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239600+0000", 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.584e-06 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239605+0000", 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6699999999999999e-06 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310166+0000", 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070560237999999997 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310173+0000", 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0389999999999999e-06 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67900 3.3 3.45adca63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.241317+0000", 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.51575353, 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.068962595000000002, 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:28.871 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67900 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.241317+0000", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.241317+0000", 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.241318+0000", 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.241329+0000", 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1707e-05 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.241336+0000", 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5610000000000004e-06 2024-05-08T03:20:28.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.241341+0000", 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9110000000000001e-06 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310273+0000", 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.068931790000000007 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310280+0000", 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2599999999999999e-06 2024-05-08T03:20:28.874 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.875 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:28.875 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:28.875 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:28.875 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:28.876 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:28.875+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:28.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:28.977 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:28.975+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:29.078 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:29.076+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:29.098 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:29.098 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:29.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.098 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82305 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.098 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096442+0000", 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00059093199999999996, 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00061484899999999999, 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82305 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096442+0000", 2024-05-08T03:20:29.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096442+0000", 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096443+0000", 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9599999999999999e-07 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096446+0000", 2024-05-08T03:20:29.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0860000000000002e-06 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096454+0000", 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2479999999999996e-06 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096461+0000", 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3169999999999998e-06 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82306 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096469+0000", 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00056448800000000003, 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000657914, 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82306 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096469+0000", 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096469+0000", 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096469+0000", 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096472+0000", 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8090000000000001e-06 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096479+0000", 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8530000000000003e-06 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096484+0000", 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5029999999999997e-06 2024-05-08T03:20:29.104 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82307 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096661+0000", 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00037192200000000001, 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00052068999999999998, 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82307 2024-05-08T03:20:29.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096661+0000", 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096661+0000", 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096662+0000", 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6499999999999998e-07 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096666+0000", 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.749e-06 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096675+0000", 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8270000000000004e-06 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096681+0000", 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2419999999999997e-06 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82308 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096691+0000", 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00034177699999999998, 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054445299999999997, 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.108 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82308 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096691+0000", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096691+0000", 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096692+0000", 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.46e-07 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096695+0000", 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096702+0000", 2024-05-08T03:20:29.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9500000000000004e-06 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096707+0000", 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3070000000000002e-06 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82301 3.9 3:9128d0b5:::rbd_data.11828f9dff47.00000000000000e7:22 [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096142+0000", 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00089137799999999996, 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011530150000000001, 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.111 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82301 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096142+0000", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096142+0000", 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096142+0000", 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096144+0000", 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0760000000000001e-06 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096151+0000", 2024-05-08T03:20:29.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7619999999999998e-06 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096156+0000", 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0640000000000001e-06 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.097235+0000", 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010789689999999999 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82302 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096163+0000", 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00087016399999999999, 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001201754, 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82302 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096163+0000", 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096163+0000", 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096163+0000", 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.01e-07 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096166+0000", 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2450000000000001e-06 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096172+0000", 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096178+0000", 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2619999999999998e-06 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82303 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096195+0000", 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00083871100000000001, 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001223643, 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.118 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82303 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096195+0000", 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096195+0000", 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096195+0000", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0299999999999999e-07 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096198+0000", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9280000000000002e-06 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096206+0000", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2020000000000004e-06 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096212+0000", 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:82304 3.9 3.ad0b1489 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:29.096401+0000", 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00063221299999999996, 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0010693530000000001, 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "queued for pg", 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:29.121 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 82304 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096401+0000", 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096401+0000", 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:29.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096402+0000", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000002e-07 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096405+0000", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.326e-06 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096415+0000", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0190999999999999e-05 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.096422+0000", 2024-05-08T03:20:29.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.81e-06 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 8 2024-05-08T03:20:29.124 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:29.125 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:29.125 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:29.126 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:29.152 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:29.152 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:29.166 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:20:29.181 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:29.180+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:29.189 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:29.189 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:29.228 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:29.228 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:29.262 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:29.262 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:29.262 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:29.280 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:29.278+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:29.295 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:29.296 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:29.331 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:29.331 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:29.339 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:29.340 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:29.341 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:29.341 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:29.353 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:29.394 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:29.393+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:29.481 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:29.480+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:29.572 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:29.572 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:29.582 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:29.581+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:29.585 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:29.684 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:29.682+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:29.785 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:29.783+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:29.810 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:29.810 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:29.819 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:29.903 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:29.902+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:29.971 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:29.971 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:29.985 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:30.027 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:30.025+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:30.088 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:30.086+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:30.138 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-05-08T03:20:30.150 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:30.150 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:30.152 INFO:teuthology.orchestra.run.smithi177.stdout:4116 trunc from 0xea5d23b to 0x3f9560a 2024-05-08T03:20:30.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:30.209 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:30.207+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:30.273 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:30.273 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:30.281 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:30.323 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:30.321+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:30.391 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:30.389+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:30.426 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:30.426 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:30.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:30.495 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:30.490+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:30.581 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:30.582 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:30.590 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:30.592 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:30.590+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:30.693 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:30.691+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:30.699 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:30.699 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:30.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:30.711 INFO:teuthology.orchestra.run.smithi177.stdout:4117 write 0xcf4ab4 thru 0xcf5f14 (0x1461 bytes) 2024-05-08T03:20:30.713 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.713 INFO:teuthology.orchestra.run.smithi177.stdout:4120 writesame 0x20647d9 thru 0x2068133 data_size 0x1(0x395b bytes) 2024-05-08T03:20:30.721 INFO:teuthology.orchestra.run.smithi177.stdout:4121 write 0x1dd9e95 thru 0x1ddd891 (0x39fd bytes) 2024-05-08T03:20:30.723 INFO:teuthology.orchestra.run.smithi177.stdout:4122 read 0xdd74bd thru 0xddbb81 (0x46c5 bytes) 2024-05-08T03:20:30.723 INFO:teuthology.orchestra.run.smithi177.stdout:4124 writesame 0x1a300c7 thru 0x1a334dd data_size 0x23(0x3417 bytes) 2024-05-08T03:20:30.744 INFO:teuthology.orchestra.run.smithi177.stdout:4125 write 0xe70459a thru 0xe7050ad (0xb14 bytes) 2024-05-08T03:20:30.747 INFO:teuthology.orchestra.run.smithi177.stdout:4126 read 0x50d5b91 thru 0x50dd3f5 (0x7865 bytes) 2024-05-08T03:20:30.747 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.748 INFO:teuthology.orchestra.run.smithi177.stdout:4128 write 0xa5508df thru 0xa55d8ce (0xcff0 bytes) 2024-05-08T03:20:30.755 INFO:teuthology.orchestra.run.smithi177.stdout:4130 read 0xe68dbce thru 0xe692c7c (0x50af bytes) 2024-05-08T03:20:30.755 INFO:teuthology.orchestra.run.smithi177.stdout:4134 write 0xb17780 thru 0xb21c7e (0xa4ff bytes) 2024-05-08T03:20:30.757 INFO:teuthology.orchestra.run.smithi177.stdout:4135 read 0x5c209a7 thru 0x5c279b3 (0x700d bytes) 2024-05-08T03:20:30.757 INFO:teuthology.orchestra.run.smithi177.stdout:4136 read 0x6999187 thru 0x699fcc2 (0x6b3c bytes) 2024-05-08T03:20:30.762 INFO:teuthology.orchestra.run.smithi177.stdout:4137 trunc from 0xe7050ae to 0xacc0db6 2024-05-08T03:20:30.778 INFO:teuthology.orchestra.run.smithi177.stdout:4139 writesame 0xe4f6bc4 thru 0xe4f973a data_size 0x1(0x2b77 bytes) 2024-05-08T03:20:30.780 INFO:teuthology.orchestra.run.smithi177.stdout:4140 read 0xb31eedf thru 0xb327bed (0x8d0f bytes) 2024-05-08T03:20:30.780 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.780 INFO:teuthology.orchestra.run.smithi177.stdout:4142 read 0x7e772 thru 0x84bbd (0x644c bytes) 2024-05-08T03:20:30.781 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.781 INFO:teuthology.orchestra.run.smithi177.stdout:4144 write 0xb183c74 thru 0xb188d91 (0x511e bytes) 2024-05-08T03:20:30.783 INFO:teuthology.orchestra.run.smithi177.stdout:4146 read 0x67ba31c thru 0x67bc0c9 (0x1dae bytes) 2024-05-08T03:20:30.784 INFO:teuthology.orchestra.run.smithi177.stdout:4147 write 0xb1676c0 thru 0xb17390f (0xc250 bytes) 2024-05-08T03:20:30.785 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.785 INFO:teuthology.orchestra.run.smithi177.stdout:4153 write 0x5052a56 thru 0x505f562 (0xcb0d bytes) 2024-05-08T03:20:30.787 INFO:teuthology.orchestra.run.smithi177.stdout:4154 punch from 0x90cfa6e to 0x90cfc9e, (0x230 bytes) 2024-05-08T03:20:30.792 INFO:teuthology.orchestra.run.smithi177.stdout:4155 write 0xe58a668 thru 0xe594f93 (0xa92c bytes) 2024-05-08T03:20:30.793 INFO:teuthology.orchestra.run.smithi177.stdout:4156 write 0xd04fdc8 thru 0xd0528fa (0x2b33 bytes) 2024-05-08T03:20:30.794 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:30.792+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:30.795 INFO:teuthology.orchestra.run.smithi177.stdout:4158 write 0xb70e573 thru 0xb71164e (0x30dc bytes) 2024-05-08T03:20:30.798 INFO:teuthology.orchestra.run.smithi177.stdout:4159 read 0xc082265 thru 0xc08cd1a (0xaab6 bytes) 2024-05-08T03:20:30.798 INFO:teuthology.orchestra.run.smithi177.stdout:4160 read 0x7053ec7 thru 0x7060321 (0xc45b bytes) 2024-05-08T03:20:30.803 INFO:teuthology.orchestra.run.smithi177.stdout:4161 write 0xe778cdd thru 0xe7886bd (0xf9e1 bytes) 2024-05-08T03:20:30.805 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.805 INFO:teuthology.orchestra.run.smithi177.stdout:4163 write 0xa81afe3 thru 0xa828196 (0xd1b4 bytes) 2024-05-08T03:20:30.806 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.814 INFO:teuthology.orchestra.run.smithi177.stdout:4166 trunc from 0xe7886be to 0x802b114 2024-05-08T03:20:30.831 INFO:teuthology.orchestra.run.smithi177.stdout:4167 punch from 0x88a44a to 0x895db3, (0xb969 bytes) 2024-05-08T03:20:30.831 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:30.831 INFO:teuthology.orchestra.run.smithi177.stdout:4169 punch from 0x41c2e4e to 0x41cae95, (0x8047 bytes) 2024-05-08T03:20:30.831 INFO:teuthology.orchestra.run.smithi177.stdout:4170 writesame 0x518fbf thru 0x51f2af data_size 0x1(0x62f1 bytes) 2024-05-08T03:20:30.851 INFO:teuthology.orchestra.run.smithi177.stdout:4171 trunc from 0x802b114 to 0x1850c09 2024-05-08T03:20:30.855 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:30.855 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:30.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:30.884 INFO:teuthology.orchestra.run.smithi177.stdout:4172 write 0xdeac605 thru 0xdeadd69 (0x1765 bytes) 2024-05-08T03:20:30.889 INFO:teuthology.orchestra.run.smithi177.stdout:4173 clone 34 order 19 su 65536 sc 3 2024-05-08T03:20:30.905 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:30.903+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:30.996 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:30.994+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:31.011 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:31.011 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:31.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:31.097 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:31.095+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:31.166 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:31.166 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:31.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:31.191 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:20:31.219 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:31.217+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:31.241 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone33 from 0xdeadd6a (overlap 0x1850c09) to 0x26e79a 2024-05-08T03:20:31.284 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:31.284 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:31.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:31.335 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:31.333+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:31.336 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-05-08T03:20:31.343 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:31.399 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:31.397+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:31.423 INFO:teuthology.orchestra.run.smithi177.stdout:4175 write 0xd95d29e thru 0xd966e3c (0x9b9f bytes) 2024-05-08T03:20:31.436 INFO:teuthology.orchestra.run.smithi177.stdout:4177 punch from 0x7e5ac6 to 0x7f34e5, (0xda1f bytes) 2024-05-08T03:20:31.440 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:31.440 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:31.441 INFO:teuthology.orchestra.run.smithi177.stdout:4178 trunc from 0xdeadd6a to 0x94dc947 2024-05-08T03:20:31.449 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:31.462 INFO:teuthology.orchestra.run.smithi177.stdout:4179 trunc from 0x94dc947 to 0xb2b85f4 2024-05-08T03:20:31.468 INFO:teuthology.orchestra.run.smithi177.stdout:4180 punch from 0x542eaa9 to 0x5434f16, (0x646d bytes) 2024-05-08T03:20:31.468 INFO:teuthology.orchestra.run.smithi177.stdout:4181 write 0x8c52a1e thru 0x8c5ccfb (0xa2de bytes) 2024-05-08T03:20:31.479 INFO:teuthology.orchestra.run.smithi177.stdout:4182 trunc from 0xb2b85f4 to 0x745902d 2024-05-08T03:20:31.499 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:31.498+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:31.500 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:31.502 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:31.502 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:31.507 INFO:teuthology.orchestra.run.smithi177.stdout:4185 trunc from 0x745902d to 0xcb0b102 2024-05-08T03:20:31.514 INFO:teuthology.orchestra.run.smithi177.stdout:4186 writesame 0xb035c65 thru 0xb038e5b data_size 0x1(0x31f7 bytes) 2024-05-08T03:20:31.524 INFO:teuthology.orchestra.run.smithi177.stdout:4188 read 0x974fc5f thru 0x9752e45 (0x31e7 bytes) 2024-05-08T03:20:31.528 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:31.528 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:31.535 INFO:teuthology.orchestra.run.smithi177.stdout:4191 trunc from 0xcb0b102 to 0x4a7cc62 2024-05-08T03:20:31.552 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:31.552 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:31.572 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:31.572 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:31.573 INFO:teuthology.orchestra.run.smithi177.stdout:4192 writesame 0xb18db1b thru 0xb19753f data_size 0x1(0x9a25 bytes) 2024-05-08T03:20:31.573 INFO:teuthology.orchestra.run.smithi177.stdout:4193 write 0x1834ff thru 0x18dac8 (0xa5ca bytes) 2024-05-08T03:20:31.582 INFO:teuthology.orchestra.run.smithi177.stdout:4194 write 0x51e41f8 thru 0x51f3210 (0xf019 bytes) 2024-05-08T03:20:31.587 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:31.588 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:31.588 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:31.588 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:31.589 INFO:teuthology.orchestra.run.smithi177.stdout:4195 read 0x88ed832 thru 0x88f3792 (0x5f61 bytes) 2024-05-08T03:20:31.589 INFO:teuthology.orchestra.run.smithi177.stdout:4198 clone 35 order 20 su 131072 sc 7 2024-05-08T03:20:31.593 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:31.593 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:31.594 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:31.597 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:31.600 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:31.598+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:31.611 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:31.612 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:31.629 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:31.629 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:31.632 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:31.632 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:31.702 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:31.701+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:31.715 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:31.715 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:31.715 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:31.715 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:31.715 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:31.724 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:31.803 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:31.802+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:31.831 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.629827173000002, 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.629817723000002, 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.836 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:31.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:31.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.629804987, 2024-05-08T03:20:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.629794410999999, 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.629785024, 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.629775844000001, 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398950808999999, 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398918571999999, 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398905241999998, 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398890268000002, 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398878959000001, 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398861289999999, 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398846091999999, 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.398831997999999, 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 55.623878025000003, 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.700984222999999, 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.700970734999999, 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.700941753999999, 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.700915148, 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.268166887, 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:31.915 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:31.917 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:31.917 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:31.915+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:32.017 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:32.016+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:32.064 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:32.064 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:32.064 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:32.064 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:32.072 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:32.118 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:32.117+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:32.176 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:20:32.219 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:32.218+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:32.225 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:32.225 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:32.226 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:32.226 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:32.226 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:32.235 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:32.247 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone34 from 0xb197540 (overlap 0x4a7cc62) to 0x4309e58 2024-05-08T03:20:32.320 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:32.319+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:32.368 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:32.368 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:32.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:32.368 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.580851068999998, 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.369 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:32.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 42.433616919999999, 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:32.375 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.243476753000003, 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.382 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:32.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.218736206999999, 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67863 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1288037190000004, 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069622329999999996, 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67863 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:32.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-07 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237344+0000", 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237349+0000", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7339999999999996e-06 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237354+0000", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.605e-06 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306659+0000", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069304893000000006 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306767+0000", 2024-05-08T03:20:32.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000108385 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306963+0000", 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00019608200000000001 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67864 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1287847429999998, 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069780320000000007, 2024-05-08T03:20:32.397 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67864 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237362+0000", 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0679999999999999e-06 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237369+0000", 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4269999999999996e-06 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237373+0000", 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.296e-06 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306990+0000", 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069617100000000001 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307041+0000", 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0796e-05 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307140+0000", 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9189000000000002e-05 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67865 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1287661379999996, 2024-05-08T03:20:32.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070114813999999998, 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67865 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:32.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.96e-07 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237381+0000", 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:20:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237386+0000", 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.48e-06 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237392+0000", 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5620000000000003e-06 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307171+0000", 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069779439999999998 2024-05-08T03:20:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307237+0000", 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5838000000000002e-05 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307493+0000", 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025623099999999999 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.405 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67866 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1287468389999997, 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070792377000000004, 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67866 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999999e-07 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237401+0000", 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7630000000000001e-06 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237407+0000", 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7169999999999996e-06 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237412+0000", 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4729999999999999e-06 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307524+0000", 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070111455000000003 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307595+0000", 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1178000000000002e-05 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308190+0000", 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000595431 2024-05-08T03:20:32.409 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67867 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1287264500000003, 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072514764999999995, 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.410 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67867 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237419+0000", 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7899999999999999e-07 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237421+0000", 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9860000000000001e-06 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237426+0000", 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.614e-06 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237431+0000", 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2680000000000002e-06 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308233+0000", 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070801136000000001 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308945+0000", 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00071273699999999996 2024-05-08T03:20:32.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309933+0000", 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0009876450000000001 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67868 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1287072809999996, 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072516667000000007, 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.414 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67868 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:32.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239091+0000", 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001654006 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239096+0000", 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.921e-06 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239107+0000", 2024-05-08T03:20:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0781000000000001e-05 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239114+0000", 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.939e-06 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309950+0000", 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070835871999999994 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309954+0000", 2024-05-08T03:20:32.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1479999999999999e-06 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67869 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1270176980000004, 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070844969999999993, 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.418 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67869 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239128+0000", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239130+0000", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.655e-06 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239138+0000", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2690000000000002e-06 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239143+0000", 2024-05-08T03:20:32.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6690000000000003e-06 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309966+0000", 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070823102999999998 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309972+0000", 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6350000000000003e-06 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67870 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.12699397, 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070850033000000007, 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67870 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.422 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239153+0000", 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2189999999999998e-06 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239160+0000", 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7170000000000002e-06 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239166+0000", 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4140000000000002e-06 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309994+0000", 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070828156000000003 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310001+0000", 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0310000000000001e-06 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67871 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.425 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1269720489999999, 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070908609999999997, 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67871 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239175+0000", 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:32.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239182+0000", 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5640000000000002e-06 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239187+0000", 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9409999999999998e-06 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310071+0000", 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070883710000000003 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310081+0000", 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0716999999999999e-05 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67876 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1268616040000001, 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070745004, 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.429 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67876 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239286+0000", 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0329999999999998e-06 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239291+0000", 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.57e-06 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239296+0000", 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5789999999999997e-06 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310022+0000", 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070725882000000004 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310028+0000", 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.432 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67877 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1268427790000004, 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070752494999999999, 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67877 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239305+0000", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239311+0000", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239316+0000", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.527e-06 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310048+0000", 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070732489999999995 2024-05-08T03:20:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310054+0000", 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2269999999999998e-06 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67878 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1268229109999996, 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070795459000000005, 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.436 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67878 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8799999999999998e-07 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239324+0000", 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239330+0000", 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8050000000000003e-06 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239335+0000", 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5029999999999999e-06 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310110+0000", 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070775530000000003 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310117+0000", 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2089999999999999e-06 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67879 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1268043990000001, 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070805241000000005, 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67879 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239341+0000", 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239343+0000", 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9400000000000001e-06 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239349+0000", 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.533e-06 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239354+0000", 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4549999999999997e-06 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310138+0000", 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070784480999999996 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310146+0000", 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67880 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:32.443 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1265550390000003, 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070583059000000004, 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67880 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2200000000000003e-07 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239593+0000", 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8059999999999999e-06 2024-05-08T03:20:32.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239600+0000", 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.584e-06 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239605+0000", 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6699999999999999e-06 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310166+0000", 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070560237999999997 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310173+0000", 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0389999999999999e-06 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:80533 3.f 3:f6536522:::rbd_data.11828c8e13f1.000000000000002f:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.43164164, 2024-05-08T03:20:32.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083900980999999999, 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 80533 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934504+0000", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8299999999999996e-07 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934509+0000", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8520000000000003e-06 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934516+0000", 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.058e-06 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934522+0000", 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2029999999999997e-06 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.935441+0000", 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000918745 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.935580+0000", 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013952300000000001 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.018404+0000", 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.082823816999999994 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:80614 3.16 3:696d6acd:::rbd_data.11828c8e13f1.0000000000000030:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 3.4260820089999999, 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.078694985999999995, 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.451 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 80614 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:32.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6699999999999999e-07 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940066+0000", 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8329999999999998e-06 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940073+0000", 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1840000000000002e-06 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940079+0000", 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7139999999999998e-06 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.943185+0000", 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0031060459999999999 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.943333+0000", 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014836 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.018758+0000", 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075424381999999998 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.455 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:32.457 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:32.458 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-05-08T03:20:32.458 INFO:teuthology.orchestra.run.smithi177.stdout:4199 writesame 0x23e7923 thru 0x23eac13 data_size 0x17(0x32f1 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4201 write 0xae05050 thru 0xae11676 (0xc627 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4202 writesame 0xdf99684 thru 0xdfa8848 data_size 0x75(0xf1c5 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4203 write 0xaf2df4e thru 0xaf33140 (0x51f3 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4204 read 0x63c5916 thru 0x63c6ff4 (0x16df bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4206 writesame 0x3447aa3 thru 0x344aef4 data_size 0x25(0x3452 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4207 read 0x9c5118 thru 0x9c5778 (0x661 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4208 writesame 0x7581f96 thru 0x758ebde data_size 0x1f(0xcc49 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4210 writesame 0x2100407 thru 0x211015f data_size 0xd(0xfd59 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4211 read 0x6e6614b thru 0x6e69915 (0x37cb bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4212 write 0x38b00f3 thru 0x38b1d40 (0x1c4e bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4213 read 0xd479447 thru 0xd482aa4 (0x965e bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4215 read 0xcb4364 thru 0xcbd39b (0x9038 bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4217 write 0xf8d456 thru 0xf95f84 (0x8b2f bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4219 read 0x9063798 thru 0x906b304 (0x7b6d bytes) 2024-05-08T03:20:32.459 INFO:teuthology.orchestra.run.smithi177.stdout:4220 writesame 0x2f7b7cf thru 0x2f7cc61 data_size 0x1(0x1493 bytes) 2024-05-08T03:20:32.460 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:32.454+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:32.461 INFO:teuthology.orchestra.run.smithi177.stdout:4221 write 0x6f079ba thru 0x6f0e79f (0x6de6 bytes) 2024-05-08T03:20:32.464 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.473 INFO:teuthology.orchestra.run.smithi177.stdout:4224 trunc from 0xdfa8849 to 0x65fbe1a 2024-05-08T03:20:32.491 INFO:teuthology.orchestra.run.smithi177.stdout:4225 read 0x27e0069 thru 0x27e3bbf (0x3b57 bytes) 2024-05-08T03:20:32.492 INFO:teuthology.orchestra.run.smithi177.stdout:4226 writesame 0x34b5dc5 thru 0x34be862 data_size 0xb(0x8a9e bytes) 2024-05-08T03:20:32.498 INFO:teuthology.orchestra.run.smithi177.stdout:4227 writesame 0x4039169 thru 0x404801b data_size 0x1(0xeeb3 bytes) 2024-05-08T03:20:32.525 INFO:teuthology.orchestra.run.smithi177.stdout:4228 read 0x4971479 thru 0x497d914 (0xc49c bytes) 2024-05-08T03:20:32.526 INFO:teuthology.orchestra.run.smithi177.stdout:4229 trunc from 0x65fbe1a to 0x6f9c9a1 2024-05-08T03:20:32.536 INFO:teuthology.orchestra.run.smithi177.stdout:4232 write 0x70c5b5e thru 0x70cf19c (0x963f bytes) 2024-05-08T03:20:32.539 INFO:teuthology.orchestra.run.smithi177.stdout:4235 punch from 0x62966c1 to 0x6296f07, (0x846 bytes) 2024-05-08T03:20:32.540 INFO:teuthology.orchestra.run.smithi177.stdout:4236 read 0x606d070 thru 0x606d180 (0x111 bytes) 2024-05-08T03:20:32.540 INFO:teuthology.orchestra.run.smithi177.stdout:4238 read 0x4edf51 thru 0x4edf79 (0x29 bytes) 2024-05-08T03:20:32.541 INFO:teuthology.orchestra.run.smithi177.stdout:4239 trunc from 0x70cf19d to 0x819dcf6 2024-05-08T03:20:32.548 INFO:teuthology.orchestra.run.smithi177.stdout:4240 read 0x459635 thru 0x45a2fa (0xcc6 bytes) 2024-05-08T03:20:32.557 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:32.555+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:32.558 INFO:teuthology.orchestra.run.smithi177.stdout:4241 write 0xc416a4c thru 0xc42545f (0xea14 bytes) 2024-05-08T03:20:32.564 INFO:teuthology.orchestra.run.smithi177.stdout:4245 read 0x613abfb thru 0x613d8d0 (0x2cd6 bytes) 2024-05-08T03:20:32.564 INFO:teuthology.orchestra.run.smithi177.stdout:4247 read 0x4a6d6ef thru 0x4a7026e (0x2b80 bytes) 2024-05-08T03:20:32.564 INFO:teuthology.orchestra.run.smithi177.stdout:4248 read 0xf180e5 thru 0xf20389 (0x82a5 bytes) 2024-05-08T03:20:32.565 INFO:teuthology.orchestra.run.smithi177.stdout:4249 write 0x46d0418 thru 0x46da182 (0x9d6b bytes) 2024-05-08T03:20:32.571 INFO:teuthology.orchestra.run.smithi177.stdout:4250 read 0x4daabe8 thru 0x4daeda5 (0x41be bytes) 2024-05-08T03:20:32.571 INFO:teuthology.orchestra.run.smithi177.stdout:4252 read 0xc120ecb thru 0xc12490a (0x3a40 bytes) 2024-05-08T03:20:32.571 INFO:teuthology.orchestra.run.smithi177.stdout:4253 read 0x5188c6b thru 0x518dd84 (0x511a bytes) 2024-05-08T03:20:32.581 INFO:teuthology.orchestra.run.smithi177.stdout:4254 trunc from 0xc425460 to 0x373dfbc 2024-05-08T03:20:32.586 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:32.586 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:32.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.586 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:99118 3.1f 3:f8cc3789:::rbd_data.1182f7c96655.0000000000000062:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T03:20:32.586 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:32.583064+0000", 2024-05-08T03:20:32.586 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.0022155069999999998, 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0022384309999999999, 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for sub ops", 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 99118 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583064+0000", 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583064+0000", 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583066+0000", 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4729999999999999e-06 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583074+0000", 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9419999999999995e-06 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583104+0000", 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0023e-05 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583114+0000", 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2969999999999992e-06 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583182+0000", 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8538e-05 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583275+0000", 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2349999999999995e-05 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583382+0000", 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010764699999999999 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:32.583931+0000", 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00054899300000000001 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.590 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.591 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.591 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:32.591 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:20:32.591 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:32.598 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:32.612 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.612 INFO:teuthology.orchestra.run.smithi177.stdout:4256 read 0x1e5bf81 thru 0x1e6578a (0x980a bytes) 2024-05-08T03:20:32.612 INFO:teuthology.orchestra.run.smithi177.stdout:4259 read 0x2fa21b0 thru 0x2fa90ee (0x6f3f bytes) 2024-05-08T03:20:32.612 INFO:teuthology.orchestra.run.smithi177.stdout:4260 punch from 0x2537782 to 0x253a87f, (0x30fd bytes) 2024-05-08T03:20:32.620 INFO:teuthology.orchestra.run.smithi177.stdout:4261 trunc from 0x373dfbc to 0xab32119 2024-05-08T03:20:32.628 INFO:teuthology.orchestra.run.smithi177.stdout:4262 writesame 0x97b26a2 thru 0x97b72f3 data_size 0x1(0x4c52 bytes) 2024-05-08T03:20:32.646 INFO:teuthology.orchestra.run.smithi177.stdout:4264 trunc from 0xab32119 to 0xeac2b82 2024-05-08T03:20:32.651 INFO:teuthology.orchestra.run.smithi177.stdout:4266 read 0xbc21b6 thru 0xbc7d19 (0x5b64 bytes) 2024-05-08T03:20:32.651 INFO:teuthology.orchestra.run.smithi177.stdout:4268 write 0xb7d3670 thru 0xb7e0fc5 (0xd956 bytes) 2024-05-08T03:20:32.657 INFO:teuthology.orchestra.run.smithi177.stdout:4269 read 0xbac1378 thru 0xbac1c0e (0x897 bytes) 2024-05-08T03:20:32.657 INFO:teuthology.orchestra.run.smithi177.stdout:4270 read 0xe6577ba thru 0xe66500b (0xd852 bytes) 2024-05-08T03:20:32.657 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.657 INFO:teuthology.orchestra.run.smithi177.stdout:4272 read 0xa7e14a0 thru 0xa7e8a73 (0x75d4 bytes) 2024-05-08T03:20:32.657 INFO:teuthology.orchestra.run.smithi177.stdout:4273 writesame 0x6083a6b thru 0x608914f data_size 0x5(0x56e5 bytes) 2024-05-08T03:20:32.658 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:32.656+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:32.662 INFO:teuthology.orchestra.run.smithi177.stdout:4275 write 0x8ec40f6 thru 0x8ec9531 (0x543c bytes) 2024-05-08T03:20:32.665 INFO:teuthology.orchestra.run.smithi177.stdout:4276 writesame 0xcd6415a thru 0xcd73a89 data_size 0x48(0xf930 bytes) 2024-05-08T03:20:32.668 INFO:teuthology.orchestra.run.smithi177.stdout:4277 write 0x9731845 thru 0x973f22f (0xd9eb bytes) 2024-05-08T03:20:32.672 INFO:teuthology.orchestra.run.smithi177.stdout:4279 writesame 0x49eae92 thru 0x49ed568 data_size 0x1(0x26d7 bytes) 2024-05-08T03:20:32.680 INFO:teuthology.orchestra.run.smithi177.stdout:4281 write 0xd79a8e8 thru 0xd7a8cd8 (0xe3f1 bytes) 2024-05-08T03:20:32.702 INFO:teuthology.orchestra.run.smithi177.stdout:4282 trunc from 0xeac2b82 to 0x21d3e45 2024-05-08T03:20:32.739 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.744 INFO:teuthology.orchestra.run.smithi177.stdout:4284 trunc from 0x21d3e45 to 0x632e5ce 2024-05-08T03:20:32.751 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:32.751 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:32.751 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:32.751 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:32.751 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:32.759 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:32.757+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:32.763 INFO:teuthology.orchestra.run.smithi177.stdout:4285 write 0xc73ab24 thru 0xc740060 (0x553d bytes) 2024-05-08T03:20:32.765 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:32.774 INFO:teuthology.orchestra.run.smithi177.stdout:4287 writesame 0x690f745 thru 0x6913543 data_size 0x1(0x3dff bytes) 2024-05-08T03:20:32.784 INFO:teuthology.orchestra.run.smithi177.stdout:4288 write 0x5f94b37 thru 0x5f97400 (0x28ca bytes) 2024-05-08T03:20:32.788 INFO:teuthology.orchestra.run.smithi177.stdout:4289 read 0x7caae3a thru 0x7cb9970 (0xeb37 bytes) 2024-05-08T03:20:32.788 INFO:teuthology.orchestra.run.smithi177.stdout:4290 punch from 0x209aed5 to 0x209bc20, (0xd4b bytes) 2024-05-08T03:20:32.788 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.788 INFO:teuthology.orchestra.run.smithi177.stdout:4293 read 0xb90c1c5 thru 0xb9104a1 (0x42dd bytes) 2024-05-08T03:20:32.788 INFO:teuthology.orchestra.run.smithi177.stdout:4294 read 0x83ea7fb thru 0x83eb00b (0x811 bytes) 2024-05-08T03:20:32.788 INFO:teuthology.orchestra.run.smithi177.stdout:4295 write 0xb90202 thru 0xba0174 (0xff73 bytes) 2024-05-08T03:20:32.799 INFO:teuthology.orchestra.run.smithi177.stdout:4296 read 0x23d839e thru 0x23e6df4 (0xea57 bytes) 2024-05-08T03:20:32.811 INFO:teuthology.orchestra.run.smithi177.stdout:4298 trunc from 0xc740061 to 0x2556161 2024-05-08T03:20:32.827 INFO:teuthology.orchestra.run.smithi177.stdout:4299 read 0x120e941 thru 0x1216e20 (0x84e0 bytes) 2024-05-08T03:20:32.843 INFO:teuthology.orchestra.run.smithi177.stdout:4300 writesame 0xb0f282b thru 0xb0f7741 data_size 0x3(0x4f17 bytes) 2024-05-08T03:20:32.843 INFO:teuthology.orchestra.run.smithi177.stdout:4301 write 0x434c998 thru 0x434ef82 (0x25eb bytes) 2024-05-08T03:20:32.846 INFO:teuthology.orchestra.run.smithi177.stdout:4302 write 0x975e74d thru 0x975f6ab (0xf5f bytes) 2024-05-08T03:20:32.850 INFO:teuthology.orchestra.run.smithi177.stdout:4303 read 0x4a2d5f3 thru 0x4a3cbaf (0xf5bd bytes) 2024-05-08T03:20:32.850 INFO:teuthology.orchestra.run.smithi177.stdout:4304 write 0x10348b7 thru 0x1043dad (0xf4f7 bytes) 2024-05-08T03:20:32.860 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:32.859+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:32.864 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.864 INFO:teuthology.orchestra.run.smithi177.stdout:4306 read 0x5356f06 thru 0x535aef4 (0x3fef bytes) 2024-05-08T03:20:32.865 INFO:teuthology.orchestra.run.smithi177.stdout:4307 write 0x50fb374 thru 0x5100ff5 (0x5c82 bytes) 2024-05-08T03:20:32.869 INFO:teuthology.orchestra.run.smithi177.stdout:4308 punch from 0x1579a95 to 0x15877ca, (0xdd35 bytes) 2024-05-08T03:20:32.869 INFO:teuthology.orchestra.run.smithi177.stdout:4309 punch from 0x3b50dcb to 0x3b5e851, (0xda86 bytes) 2024-05-08T03:20:32.869 INFO:teuthology.orchestra.run.smithi177.stdout:4312 read 0xc1f1fb thru 0xc2ea66 (0xf86c bytes) 2024-05-08T03:20:32.882 INFO:teuthology.orchestra.run.smithi177.stdout:4313 trunc from 0xb0f7742 to 0x4ad35e 2024-05-08T03:20:32.905 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:32.905 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.803250788, 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.910 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.801530318000001, 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.801496877000002, 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.915 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:32.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:32.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.800071138, 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.799978524, 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.924 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.799467849999999, 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:32.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.799440342, 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.933 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.799033910999999, 2024-05-08T03:20:32.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:32.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:32.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:32.941 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.798578533000001, 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:32.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:32.946 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.798050285999999, 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:32.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:32.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797577788000002, 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:32.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.955 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797499655999999, 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:32.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:32.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:32.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797424234000001, 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:32.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:32.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.964 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797347071000001, 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797269547999999, 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:32.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:32.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:32.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.973 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797242636, 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:32.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797221426, 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:32.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.797201646000001, 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:32.982 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:32.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:32.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.796647119999999, 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.987 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:32.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.796628108, 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:32.991 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:32.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:32.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:32.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:32.995 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:32.996 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:32.997 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4316 punch from 0x2c70f to 0x38efd, (0xc7ee bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4318 read 0x2133a5 thru 0x214136 (0xd92 bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4320 read 0x2414cd thru 0x245817 (0x434b bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4321 write 0x8288c93 thru 0x828ab64 (0x1ed2 bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4322 writesame 0xe54cae5 thru 0xe553376 data_size 0x5(0x6892 bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4323 write 0x7effcf7 thru 0x7f08d22 (0x902c bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4331 write 0xbeaa7b3 thru 0xbeb5028 (0xa876 bytes) 2024-05-08T03:20:32.999 INFO:teuthology.orchestra.run.smithi177.stdout:4332 read 0x69ca69d thru 0x69d0ac6 (0x642a bytes) 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4333 read 0x3e4ba7f thru 0x3e55e92 (0xa414 bytes) 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4334 read 0xaee7e24 thru 0xaef0f7e (0x915b bytes) 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4337 write 0x759944e thru 0x75a77ea (0xe39d bytes) 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4338 write 0x788f757 thru 0x789cb83 (0xd42d bytes) 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4340 writesame 0xb44dfe1 thru 0xb450d0e data_size 0x1(0x2d2e bytes) 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4341 flatten 2024-05-08T03:20:33.000 INFO:teuthology.orchestra.run.smithi177.stdout:4342 writesame 0x7bc8f28 thru 0x7bd0d77 data_size 0x5e(0x7e50 bytes) 2024-05-08T03:20:33.000 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:32.995+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:33.002 INFO:teuthology.orchestra.run.smithi177.stdout:4343 write 0x4bc377f thru 0x4bcde21 (0xa6a3 bytes) 2024-05-08T03:20:33.005 INFO:teuthology.orchestra.run.smithi177.stdout:4345 write 0xd46250c thru 0xd470954 (0xe449 bytes) 2024-05-08T03:20:33.008 INFO:teuthology.orchestra.run.smithi177.stdout:4346 punch from 0x767d574 to 0x768bad8, (0xe564 bytes) 2024-05-08T03:20:33.008 INFO:teuthology.orchestra.run.smithi177.stdout:4347 read 0xa75509d thru 0xa75d2e1 (0x8245 bytes) 2024-05-08T03:20:33.008 INFO:teuthology.orchestra.run.smithi177.stdout:4348 write 0x6bb90b4 thru 0x6bc149d (0x83ea bytes) 2024-05-08T03:20:33.015 INFO:teuthology.orchestra.run.smithi177.stdout:4349 read 0x193fe83 thru 0x194536f (0x54ed bytes) 2024-05-08T03:20:33.015 INFO:teuthology.orchestra.run.smithi177.stdout:4350 read 0xcd503b thru 0xcdbab6 (0x6a7c bytes) 2024-05-08T03:20:33.015 INFO:teuthology.orchestra.run.smithi177.stdout:4351 punch from 0xc895321 to 0xc8a4b65, (0xf844 bytes) 2024-05-08T03:20:33.015 INFO:teuthology.orchestra.run.smithi177.stdout:4352 trunc from 0xe553377 to 0xe4b9c63 2024-05-08T03:20:33.022 INFO:teuthology.orchestra.run.smithi177.stdout:4353 write 0x1e5ad9f thru 0x1e61681 (0x68e3 bytes) 2024-05-08T03:20:33.026 INFO:teuthology.orchestra.run.smithi177.stdout:4354 read 0x5d959e1 thru 0x5da333c (0xd95c bytes) 2024-05-08T03:20:33.026 INFO:teuthology.orchestra.run.smithi177.stdout:4356 read 0x7b6560a thru 0x7b71194 (0xbb8b bytes) 2024-05-08T03:20:33.026 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.033 INFO:teuthology.orchestra.run.smithi177.stdout:4358 trunc from 0xe4b9c63 to 0x7ab26d4 2024-05-08T03:20:33.046 INFO:teuthology.orchestra.run.smithi177.stdout:4360 punch from 0x74a0b3f to 0x74ae63e, (0xdaff bytes) 2024-05-08T03:20:33.046 INFO:teuthology.orchestra.run.smithi177.stdout:4361 write 0x5006576 thru 0x500efb5 (0x8a40 bytes) 2024-05-08T03:20:33.049 INFO:teuthology.orchestra.run.smithi177.stdout:4364 read 0x4442e87 thru 0x445141c (0xe596 bytes) 2024-05-08T03:20:33.049 INFO:teuthology.orchestra.run.smithi177.stdout:4365 read 0x32d1046 thru 0x32d39b1 (0x296c bytes) 2024-05-08T03:20:33.049 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.049 INFO:teuthology.orchestra.run.smithi177.stdout:4367 write 0x42a7386 thru 0x42ab9ae (0x4629 bytes) 2024-05-08T03:20:33.056 INFO:teuthology.orchestra.run.smithi177.stdout:4368 trunc from 0x7ab26d4 to 0x42dd7af 2024-05-08T03:20:33.071 INFO:teuthology.orchestra.run.smithi177.stdout:4369 trunc from 0x42dd7af to 0xf1ff2f 2024-05-08T03:20:33.097 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:33.095+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:33.098 INFO:teuthology.orchestra.run.smithi177.stdout:4370 write 0xade63f6 thru 0xadf06ae (0xa2b9 bytes) 2024-05-08T03:20:33.101 INFO:teuthology.orchestra.run.smithi177.stdout:4371 writesame 0x37a2573 thru 0x37a3c0e data_size 0x2(0x169c bytes) 2024-05-08T03:20:33.107 INFO:teuthology.orchestra.run.smithi177.stdout:4372 write 0x2e91b98 thru 0x2e98436 (0x689f bytes) 2024-05-08T03:20:33.110 INFO:teuthology.orchestra.run.smithi177.stdout:4373 write 0xa73466 thru 0xa73f82 (0xb1d bytes) 2024-05-08T03:20:33.113 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.113 INFO:teuthology.orchestra.run.smithi177.stdout:4376 write 0x23c9f11 thru 0x23d91f2 (0xf2e2 bytes) 2024-05-08T03:20:33.122 INFO:teuthology.orchestra.run.smithi177.stdout:4377 write 0x63798df thru 0x637e6f7 (0x4e19 bytes) 2024-05-08T03:20:33.125 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:33.125 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:33.125 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:33.125 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:33.126 INFO:teuthology.orchestra.run.smithi177.stdout:4378 read 0x5101dae thru 0x510a148 (0x839b bytes) 2024-05-08T03:20:33.126 INFO:teuthology.orchestra.run.smithi177.stdout:4379 read 0xa493926 thru 0xa49713f (0x381a bytes) 2024-05-08T03:20:33.127 INFO:teuthology.orchestra.run.smithi177.stdout:4380 punch from 0x99e5fa4 to 0x99f465d, (0xe6b9 bytes) 2024-05-08T03:20:33.135 INFO:teuthology.orchestra.run.smithi177.stdout:4381 trunc from 0xadf06af to 0x431459c 2024-05-08T03:20:33.138 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:33.155 INFO:teuthology.orchestra.run.smithi177.stdout:4382 write 0x85e9216 thru 0x85f4a50 (0xb83b bytes) 2024-05-08T03:20:33.164 INFO:teuthology.orchestra.run.smithi177.stdout:4383 trunc from 0x85f4a51 to 0xd23c3b6 2024-05-08T03:20:33.170 INFO:teuthology.orchestra.run.smithi177.stdout:4384 write 0x39efa4c thru 0x39f9208 (0x97bd bytes) 2024-05-08T03:20:33.173 INFO:teuthology.orchestra.run.smithi177.stdout:4385 write 0x3f43951 thru 0x3f4590d (0x1fbd bytes) 2024-05-08T03:20:33.177 INFO:teuthology.orchestra.run.smithi177.stdout:4386 read 0x4712161 thru 0x47184ad (0x634d bytes) 2024-05-08T03:20:33.177 INFO:teuthology.orchestra.run.smithi177.stdout:4388 punch from 0x9b2443b to 0x9b2518a, (0xd4f bytes) 2024-05-08T03:20:33.177 INFO:teuthology.orchestra.run.smithi177.stdout:4389 read 0x697444d thru 0x6975c6e (0x1822 bytes) 2024-05-08T03:20:33.188 INFO:teuthology.orchestra.run.smithi177.stdout:4390 trunc from 0xd23c3b6 to 0x4263823 2024-05-08T03:20:33.198 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:33.197+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:33.198 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.199 INFO:teuthology.orchestra.run.smithi177.stdout:4393 trunc from 0x4263823 to 0x39f7de7 2024-05-08T03:20:33.223 INFO:teuthology.orchestra.run.smithi177.stdout:4394 writesame 0x80eb68d thru 0x80f6bee data_size 0x6(0xb562 bytes) 2024-05-08T03:20:33.223 INFO:teuthology.orchestra.run.smithi177.stdout:4395 read 0x53decd1 thru 0x53edc91 (0xefc1 bytes) 2024-05-08T03:20:33.232 INFO:teuthology.orchestra.run.smithi177.stdout:4397 trunc from 0x80f6bef to 0x7ad57e 2024-05-08T03:20:33.264 INFO:teuthology.orchestra.run.smithi177.stdout:4398 write 0x70b64d9 thru 0x70c45cc (0xe0f4 bytes) 2024-05-08T03:20:33.269 INFO:teuthology.orchestra.run.smithi177.stdout:4399 write 0x4bdec22 thru 0x4be25ab (0x398a bytes) 2024-05-08T03:20:33.273 INFO:teuthology.orchestra.run.smithi177.stdout:4403 writesame 0x56909e3 thru 0x5698c34 data_size 0x7(0x8252 bytes) 2024-05-08T03:20:33.279 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:33.279 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:33.279 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:33.279 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:33.279 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:33.290 INFO:teuthology.orchestra.run.smithi177.stdout:4404 write 0xbcd383d thru 0xbcd5a52 (0x2216 bytes) 2024-05-08T03:20:33.292 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:33.293 INFO:teuthology.orchestra.run.smithi177.stdout:4406 punch from 0x9f9fbd9 to 0x9fa3725, (0x3b4c bytes) 2024-05-08T03:20:33.294 INFO:teuthology.orchestra.run.smithi177.stdout:4407 read 0xa8ff2b2 thru 0xa9018d6 (0x2625 bytes) 2024-05-08T03:20:33.294 INFO:teuthology.orchestra.run.smithi177.stdout:4408 writesame 0x9ca3164 thru 0x9ca7aee data_size 0x1(0x498b bytes) 2024-05-08T03:20:33.299 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:33.298+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:33.305 INFO:teuthology.orchestra.run.smithi177.stdout:4409 writesame 0x81bc714 thru 0x81bfb44 data_size 0x1(0x3431 bytes) 2024-05-08T03:20:33.314 INFO:teuthology.orchestra.run.smithi177.stdout:4410 write 0xa4530df thru 0xa459685 (0x65a7 bytes) 2024-05-08T03:20:33.318 INFO:teuthology.orchestra.run.smithi177.stdout:4411 read 0x80fc4dc thru 0x8102503 (0x6028 bytes) 2024-05-08T03:20:33.318 INFO:teuthology.orchestra.run.smithi177.stdout:4412 write 0xb3e9ddb thru 0xb3ed591 (0x37b7 bytes) 2024-05-08T03:20:33.321 INFO:teuthology.orchestra.run.smithi177.stdout:4414 read 0xb76ced7 thru 0xb775c78 (0x8da2 bytes) 2024-05-08T03:20:33.321 INFO:teuthology.orchestra.run.smithi177.stdout:4415 read 0x3962b38 thru 0x3963de4 (0x12ad bytes) 2024-05-08T03:20:33.321 INFO:teuthology.orchestra.run.smithi177.stdout:4417 writesame 0x74dd868 thru 0x74ec91b data_size 0xf(0xf0b4 bytes) 2024-05-08T03:20:33.326 INFO:teuthology.orchestra.run.smithi177.stdout:4419 punch from 0x7487770 to 0x748fe08, (0x8698 bytes) 2024-05-08T03:20:33.326 INFO:teuthology.orchestra.run.smithi177.stdout:4420 write 0x2cae6f5 thru 0x2cb4340 (0x5c4c bytes) 2024-05-08T03:20:33.330 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.330 INFO:teuthology.orchestra.run.smithi177.stdout:4425 write 0x6a883f2 thru 0x6a9626b (0xde7a bytes) 2024-05-08T03:20:33.333 INFO:teuthology.orchestra.run.smithi177.stdout:4426 read 0xe93bcc thru 0xe96541 (0x2976 bytes) 2024-05-08T03:20:33.333 INFO:teuthology.orchestra.run.smithi177.stdout:4427 read 0x9eb3efa thru 0x9eb7d42 (0x3e49 bytes) 2024-05-08T03:20:33.333 INFO:teuthology.orchestra.run.smithi177.stdout:4428 write 0xb8cfb63 thru 0xb8da710 (0xabae bytes) 2024-05-08T03:20:33.336 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6] out_osds: [4, 7, 0, 5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:33.336 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:33.337 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-05-08T03:20:33.337 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-05-08T03:20:33.338 INFO:teuthology.orchestra.run.smithi177.stdout:4432 punch from 0x3070359 to 0x307c9be, (0xc665 bytes) 2024-05-08T03:20:33.338 INFO:teuthology.orchestra.run.smithi177.stdout:4434 write 0xa72f73 thru 0xa7b1f9 (0x8287 bytes) 2024-05-08T03:20:33.343 INFO:teuthology.orchestra.run.smithi177.stdout:4435 writesame 0x95247e thru 0x95c08f data_size 0x3(0x9c12 bytes) 2024-05-08T03:20:33.350 INFO:teuthology.orchestra.run.smithi177.stdout:4436 write 0x95dc30c thru 0x95e238d (0x6082 bytes) 2024-05-08T03:20:33.355 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.355 INFO:teuthology.orchestra.run.smithi177.stdout:4438 read 0xb5f7644 thru 0xb5f8146 (0xb03 bytes) 2024-05-08T03:20:33.355 INFO:teuthology.orchestra.run.smithi177.stdout:4439 punch from 0x82d149c to 0x82d8ffc, (0x7b60 bytes) 2024-05-08T03:20:33.355 INFO:teuthology.orchestra.run.smithi177.stdout:4440 write 0x32a91b4 thru 0x32a9d4f (0xb9c bytes) 2024-05-08T03:20:33.358 INFO:teuthology.orchestra.run.smithi177.stdout:4441 write 0x3daa9b4 thru 0x3dac06c (0x16b9 bytes) 2024-05-08T03:20:33.372 INFO:teuthology.orchestra.run.smithi177.stdout:4442 write 0xecfb084 thru 0xed06c4c (0xbbc9 bytes) 2024-05-08T03:20:33.376 INFO:teuthology.orchestra.run.smithi177.stdout:4443 read 0x4e3715d thru 0x4e3ab84 (0x3a28 bytes) 2024-05-08T03:20:33.376 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.376 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.377 INFO:teuthology.orchestra.run.smithi177.stdout:4446 punch from 0x5c19eaf to 0x5c1e2bc, (0x440d bytes) 2024-05-08T03:20:33.377 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.377 INFO:teuthology.orchestra.run.smithi177.stdout:4449 punch from 0xe936399 to 0xe938827, (0x248e bytes) 2024-05-08T03:20:33.377 INFO:teuthology.orchestra.run.smithi177.stdout:4450 writesame 0x37a4bb6 thru 0x37b2e33 data_size 0x35(0xe27e bytes) 2024-05-08T03:20:33.394 INFO:teuthology.orchestra.run.smithi177.stdout:4451 trunc from 0xed06c4d to 0x737ec28 2024-05-08T03:20:33.401 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:33.399+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:33.420 INFO:teuthology.orchestra.run.smithi177.stdout:4452 write 0x3211d21 thru 0x3221461 (0xf741 bytes) 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:33.423 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.424544441000002, 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.424476976999998, 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.429 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:33.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:33.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.364642645000004, 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:33.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:33.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:33.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.441 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.298134632, 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.442 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:33.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.271596985999999, 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:33.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:33.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:33.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:33.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:33.452 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.243685218000003, 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1651050249999999, 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:20:33.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:20:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71203 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 561152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1604975140000002, 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075437740000000003, 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.461 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71203 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260428+0000", 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260435+0000", 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3260000000000001e-06 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260440+0000", 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8189999999999999e-06 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335711+0000", 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075271494999999994 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335784+0000", 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2786000000000004e-05 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335864+0000", 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9684999999999993e-05 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1604776489999997, 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.465 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:20:33.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:20:33.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1604570140000003, 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:20:33.469 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:20:33.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1604347629999996, 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.473 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:33.477 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1604143320000002, 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.160394106, 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:33.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:20:33.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:20:33.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1603733119999999, 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 5.1603472979999996, 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:20:33.490 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:20:33.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9803903780000001, 2024-05-08T03:20:33.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9803712740000003, 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:20:33.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:20:33.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.9798096999999997, 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:20:33.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.506 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.974595839, 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:20:33.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.511 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.974428498, 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:33.512 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:20:33.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:20:33.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:20:33.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:33.517 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:33.518 INFO:teuthology.orchestra.run.smithi177.stdout:4454 punch from 0x36a2bdd to 0x36aa3dc, (0x77ff bytes) 2024-05-08T03:20:33.519 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:33.520 INFO:teuthology.orchestra.run.smithi177.stdout:4456 write 0xa75e05b thru 0xa75e805 (0x7ab bytes) 2024-05-08T03:20:33.520 INFO:teuthology.orchestra.run.smithi177.stdout:4457 writesame 0x69c357c thru 0x69ce319 data_size 0x47(0xad9e bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4458 read 0x96e886e thru 0x96e955a (0xced bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4459 read 0x46304e5 thru 0x463284c (0x2368 bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4460 write 0x57b4f1a thru 0x57b61d7 (0x12be bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4462 write 0xacb5682 thru 0xacc2ef6 (0xd875 bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4463 writesame 0xe3223a0 thru 0xe326885 data_size 0x1(0x44e6 bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4466 punch from 0x5a552be to 0x5a5bd3a, (0x6a7c bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4467 read 0xb92470f thru 0xb930c2d (0xc51f bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4469 punch from 0x3aa5d04 to 0x3ab4fa1, (0xf29d bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4470 trunc from 0xe326886 to 0xd36521a 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4472 read 0x5e0c71 thru 0x5e6cb1 (0x6041 bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4473 trunc from 0xd36521a to 0x40db2bd 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4475 punch from 0x387e34f to 0x387f74f, (0x1400 bytes) 2024-05-08T03:20:33.521 INFO:teuthology.orchestra.run.smithi177.stdout:4477 trunc from 0x40db2bd to 0x104881 2024-05-08T03:20:33.522 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:33.516+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:33.531 INFO:teuthology.orchestra.run.smithi177.stdout:4478 punch from 0xe522f to 0xea554, (0x5325 bytes) 2024-05-08T03:20:33.531 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.541 INFO:teuthology.orchestra.run.smithi177.stdout:4480 write 0x1381d8e thru 0x1389d2c (0x7f9f bytes) 2024-05-08T03:20:33.544 INFO:teuthology.orchestra.run.smithi177.stdout:4481 read 0xb75df thru 0xbf0d8 (0x7afa bytes) 2024-05-08T03:20:33.553 INFO:teuthology.orchestra.run.smithi177.stdout:4482 write 0x44d09fc thru 0x44d4e22 (0x4427 bytes) 2024-05-08T03:20:33.556 INFO:teuthology.orchestra.run.smithi177.stdout:4487 read 0x384409d thru 0x3852304 (0xe268 bytes) 2024-05-08T03:20:33.556 INFO:teuthology.orchestra.run.smithi177.stdout:4489 read 0x40167e4 thru 0x4020f90 (0xa7ad bytes) 2024-05-08T03:20:33.573 INFO:teuthology.orchestra.run.smithi177.stdout:4491 write 0xdcdf070 thru 0xdce1f31 (0x2ec2 bytes) 2024-05-08T03:20:33.577 INFO:teuthology.orchestra.run.smithi177.stdout:4493 read 0x473eae5 thru 0x473edaa (0x2c6 bytes) 2024-05-08T03:20:33.577 INFO:teuthology.orchestra.run.smithi177.stdout:4494 write 0x7c6ed67 thru 0x7c7e139 (0xf3d3 bytes) 2024-05-08T03:20:33.580 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.586 INFO:teuthology.orchestra.run.smithi177.stdout:4496 write 0xe4e11c0 thru 0xe4e9b03 (0x8944 bytes) 2024-05-08T03:20:33.589 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.589 INFO:teuthology.orchestra.run.smithi177.stdout:4499 read 0x1d57dc2 thru 0x1d60cf0 (0x8f2f bytes) 2024-05-08T03:20:33.589 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.589 INFO:teuthology.orchestra.run.smithi177.stdout:4504 punch from 0x2d9eb36 to 0x2da357f, (0x4a49 bytes) 2024-05-08T03:20:33.589 INFO:teuthology.orchestra.run.smithi177.stdout:4506 write 0x30002d2 thru 0x3007b56 (0x7885 bytes) 2024-05-08T03:20:33.597 INFO:teuthology.orchestra.run.smithi177.stdout:4508 writesame 0xe9bacf3 thru 0xe9c73b9 data_size 0x1(0xc6c7 bytes) 2024-05-08T03:20:33.597 INFO:teuthology.orchestra.run.smithi177.stdout:4509 read 0x18ec6a7 thru 0x18edf8d (0x18e7 bytes) 2024-05-08T03:20:33.597 INFO:teuthology.orchestra.run.smithi177.stdout:4510 punch from 0x3b14153 to 0x3b183bc, (0x4269 bytes) 2024-05-08T03:20:33.597 INFO:teuthology.orchestra.run.smithi177.stdout:4511 write 0x2da1a98 thru 0x2da3f02 (0x246b bytes) 2024-05-08T03:20:33.600 INFO:teuthology.orchestra.run.smithi177.stdout:4513 read 0xc02d71a thru 0xc03066a (0x2f51 bytes) 2024-05-08T03:20:33.600 INFO:teuthology.orchestra.run.smithi177.stdout:4514 read 0x51a7bd5 thru 0x51a89d2 (0xdfe bytes) 2024-05-08T03:20:33.600 INFO:teuthology.orchestra.run.smithi177.stdout:4515 read 0x909bb7d thru 0x909fe16 (0x429a bytes) 2024-05-08T03:20:33.600 INFO:teuthology.orchestra.run.smithi177.stdout:4517 read 0xd431d2c thru 0xd433597 (0x186c bytes) 2024-05-08T03:20:33.600 INFO:teuthology.orchestra.run.smithi177.stdout:4519 punch from 0x1c9fd1 to 0x1d06ec, (0x671b bytes) 2024-05-08T03:20:33.609 INFO:teuthology.orchestra.run.smithi177.stdout:4520 trunc from 0xe9c73ba to 0x7e9bf47 2024-05-08T03:20:33.619 INFO:teuthology.orchestra.run.smithi177.stdout:4521 read 0x35d1570 thru 0x35d9c84 (0x8715 bytes) 2024-05-08T03:20:33.619 INFO:teuthology.orchestra.run.smithi177.stdout:4523 read 0x34cfe0f thru 0x34d4e82 (0x5074 bytes) 2024-05-08T03:20:33.619 INFO:teuthology.orchestra.run.smithi177.stdout:4525 read 0x7385cc7 thru 0x7394966 (0xeca0 bytes) 2024-05-08T03:20:33.619 INFO:teuthology.orchestra.run.smithi177.stdout:4526 write 0x39befe0 thru 0x39c2282 (0x32a3 bytes) 2024-05-08T03:20:33.619 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:33.617+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:33.622 INFO:teuthology.orchestra.run.smithi177.stdout:4527 punch from 0x4095052 to 0x40a0b5e, (0xbb0c bytes) 2024-05-08T03:20:33.622 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.622 INFO:teuthology.orchestra.run.smithi177.stdout:4529 punch from 0x3382039 to 0x3383963, (0x192a bytes) 2024-05-08T03:20:33.622 INFO:teuthology.orchestra.run.smithi177.stdout:4530 write 0x2d1fe8a thru 0x2d28f45 (0x90bc bytes) 2024-05-08T03:20:33.625 INFO:teuthology.orchestra.run.smithi177.stdout:4531 read 0x6daeb42 thru 0x6dbc703 (0xdbc2 bytes) 2024-05-08T03:20:33.632 INFO:teuthology.orchestra.run.smithi177.stdout:4534 write 0x9d22501 thru 0x9d2f60d (0xd10d bytes) 2024-05-08T03:20:33.633 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:33.634 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:33.634 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:33.634 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:33.634 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:33.637 INFO:teuthology.orchestra.run.smithi177.stdout:4535 read 0x7480932 thru 0x748acbe (0xa38d bytes) 2024-05-08T03:20:33.643 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:33.645 INFO:teuthology.orchestra.run.smithi177.stdout:4536 trunc from 0x9d2f60e to 0x32f7161 2024-05-08T03:20:33.656 INFO:teuthology.orchestra.run.smithi177.stdout:4537 read 0x2b19a0c thru 0x2b2155c (0x7b51 bytes) 2024-05-08T03:20:33.671 INFO:teuthology.orchestra.run.smithi177.stdout:4539 trunc from 0x32f7161 to 0xdee4205 2024-05-08T03:20:33.690 INFO:teuthology.orchestra.run.smithi177.stdout:4541 trunc from 0xdee4205 to 0x2450424 2024-05-08T03:20:33.711 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:20:33.710+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e80 definitely_dead 0 2024-05-08T03:20:33.714 INFO:teuthology.orchestra.run.smithi177.stdout:4542 write 0x8c6f933 thru 0x8c73070 (0x373e bytes) 2024-05-08T03:20:33.720 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:33.719+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:33.728 INFO:teuthology.orchestra.run.smithi177.stdout:4544 writesame 0xc3e4ac9 thru 0xc3e6f35 data_size 0x5(0x246d bytes) 2024-05-08T03:20:33.728 INFO:teuthology.orchestra.run.smithi177.stdout:4545 punch from 0xa2d4a62 to 0xa2dfdea, (0xb388 bytes) 2024-05-08T03:20:33.728 INFO:teuthology.orchestra.run.smithi177.stdout:4546 write 0xa91fe1e thru 0xa92a4b5 (0xa698 bytes) 2024-05-08T03:20:33.734 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.746 INFO:teuthology.orchestra.run.smithi177.stdout:4549 trunc from 0xc3e6f36 to 0x2229781 2024-05-08T03:20:33.760 INFO:teuthology.orchestra.run.smithi177.stdout:4551 read 0x146ec30 thru 0x1475558 (0x6929 bytes) 2024-05-08T03:20:33.773 INFO:teuthology.orchestra.run.smithi177.stdout:4554 writesame 0x776c36d thru 0x776e402 data_size 0x1(0x2096 bytes) 2024-05-08T03:20:33.781 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:33.781 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:33.786 INFO:teuthology.orchestra.run.smithi177.stdout:4555 writesame 0xdb2eed7 thru 0xdb3ea08 data_size 0x4f(0xfb32 bytes) 2024-05-08T03:20:33.790 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:33.790 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:33.790 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:33.790 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:33.790 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:33.794 INFO:teuthology.orchestra.run.smithi177.stdout:4556 trunc from 0xdb3ea09 to 0x780914a 2024-05-08T03:20:33.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:33.800 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:33.801 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:33.814 INFO:teuthology.orchestra.run.smithi177.stdout:4560 writesame 0xcadd130 thru 0xcaddb70 data_size 0x19(0xa41 bytes) 2024-05-08T03:20:33.814 INFO:teuthology.orchestra.run.smithi177.stdout:4562 write 0x536af76 thru 0x5376f5a (0xbfe5 bytes) 2024-05-08T03:20:33.818 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.818 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.818 INFO:teuthology.orchestra.run.smithi177.stdout:4567 write 0x61f02b1 thru 0x61f105c (0xdac bytes) 2024-05-08T03:20:33.819 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:33.819 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:33.820 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:33.819+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:33.822 INFO:teuthology.orchestra.run.smithi177.stdout:4569 read 0x12d696d thru 0x12d6cb2 (0x346 bytes) 2024-05-08T03:20:33.822 INFO:teuthology.orchestra.run.smithi177.stdout:4570 write 0xbae2917 thru 0xbaea9c9 (0x80b3 bytes) 2024-05-08T03:20:33.825 INFO:teuthology.orchestra.run.smithi177.stdout:4571 writesame 0xa2c2746 thru 0xa2c7e90 data_size 0x9(0x574b bytes) 2024-05-08T03:20:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:4572 punch from 0x74b4c6c to 0x74b8edd, (0x4271 bytes) 2024-05-08T03:20:33.829 INFO:teuthology.orchestra.run.smithi177.stdout:4573 punch from 0x3d0cd3e to 0x3d17dbf, (0xb081 bytes) 2024-05-08T03:20:33.829 INFO:teuthology.orchestra.run.smithi177.stdout:4574 writesame 0x447c526 thru 0x447f963 data_size 0x9(0x343e bytes) 2024-05-08T03:20:33.832 INFO:teuthology.orchestra.run.smithi177.stdout:4575 punch from 0x8a14c21 to 0x8a1a03f, (0x541e bytes) 2024-05-08T03:20:33.832 INFO:teuthology.orchestra.run.smithi177.stdout:4576 read 0x5cbd5e4 thru 0x5cc7065 (0x9a82 bytes) 2024-05-08T03:20:33.832 INFO:teuthology.orchestra.run.smithi177.stdout:4577 punch from 0x9990d3e to 0x999632f, (0x55f1 bytes) 2024-05-08T03:20:33.832 INFO:teuthology.orchestra.run.smithi177.stdout:4578 writesame 0x44c4651 thru 0x44cb05b data_size 0x1(0x6a0b bytes) 2024-05-08T03:20:33.838 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:33.839 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:33.848 INFO:teuthology.orchestra.run.smithi177.stdout:4579 read 0x97fb1b1 thru 0x9801d03 (0x6b53 bytes) 2024-05-08T03:20:33.848 INFO:teuthology.orchestra.run.smithi177.stdout:4580 punch from 0x3bd91d2 to 0x3be15e3, (0x8411 bytes) 2024-05-08T03:20:33.854 INFO:teuthology.orchestra.run.smithi177.stdout:4581 write 0xdc40019 thru 0xdc412cd (0x12b5 bytes) 2024-05-08T03:20:33.857 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:33.857 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:33.857 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:33.862 INFO:teuthology.orchestra.run.smithi177.stdout:4582 trunc from 0xdc412ce to 0x99983fe 2024-05-08T03:20:33.872 INFO:teuthology.orchestra.run.smithi177.stdout:4583 write 0x3581ac4 thru 0x358523f (0x377c bytes) 2024-05-08T03:20:33.875 INFO:teuthology.orchestra.run.smithi177.stdout:4584 read 0x1614dd3 thru 0x1615d58 (0xf86 bytes) 2024-05-08T03:20:33.875 INFO:teuthology.orchestra.run.smithi177.stdout:4585 writesame 0x23fd3fb thru 0x2408ebf data_size 0x1(0xbac5 bytes) 2024-05-08T03:20:33.877 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:33.877 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:33.895 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:33.895 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:33.898 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:33.899 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:33.903 INFO:teuthology.orchestra.run.smithi177.stdout:4588 write 0x49fbf6c thru 0x4a04a33 (0x8ac8 bytes) 2024-05-08T03:20:33.909 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:33.909 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:33.909 INFO:teuthology.orchestra.run.smithi177.stdout:4589 writesame 0x38e5b0f thru 0x38f496e data_size 0x10(0xee60 bytes) 2024-05-08T03:20:33.914 INFO:teuthology.orchestra.run.smithi177.stdout:4590 write 0xb9e9e9 thru 0xba3198 (0x47b0 bytes) 2024-05-08T03:20:33.918 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:33.919 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.921 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:33.920+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:33.928 INFO:teuthology.orchestra.run.smithi177.stdout:4593 writesame 0xd3f0029 thru 0xd3f80cd data_size 0x1(0x80a5 bytes) 2024-05-08T03:20:33.928 INFO:teuthology.orchestra.run.smithi177.stdout:4594 punch from 0xd4d99a to 0xd56146, (0x87ac bytes) 2024-05-08T03:20:33.928 INFO:teuthology.orchestra.run.smithi177.stdout:4595 write 0xc425110 thru 0xc42d05c (0x7f4d bytes) 2024-05-08T03:20:33.931 INFO:teuthology.orchestra.run.smithi177.stdout:4596 read 0xc40e93f thru 0xc419a13 (0xb0d5 bytes) 2024-05-08T03:20:33.931 INFO:teuthology.orchestra.run.smithi177.stdout:4598 writesame 0x4187b39 thru 0x418ec76 data_size 0x5(0x713e bytes) 2024-05-08T03:20:33.941 INFO:teuthology.orchestra.run.smithi177.stdout:4599 trunc from 0xd3f80ce to 0x93f24ea 2024-05-08T03:20:33.950 INFO:teuthology.orchestra.run.smithi177.stdout:4601 punch from 0x439a565 to 0x43a4d42, (0xa7dd bytes) 2024-05-08T03:20:33.950 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:33.951 INFO:teuthology.orchestra.run.smithi177.stdout:4603 read 0x284622 thru 0x285e7a (0x1859 bytes) 2024-05-08T03:20:33.951 INFO:teuthology.orchestra.run.smithi177.stdout:4604 read 0x50cd602 thru 0x50d7fe9 (0xa9e8 bytes) 2024-05-08T03:20:33.951 INFO:teuthology.orchestra.run.smithi177.stdout:4605 writesame 0x137a427 thru 0x137ae45 data_size 0x1(0xa1f bytes) 2024-05-08T03:20:33.954 INFO:teuthology.orchestra.run.smithi177.stdout:4606 read 0x87eca24 thru 0x87f0bfe (0x41db bytes) 2024-05-08T03:20:33.954 INFO:teuthology.orchestra.run.smithi177.stdout:4607 read 0x72d2b94 thru 0x72dd4ce (0xa93b bytes) 2024-05-08T03:20:33.954 INFO:teuthology.orchestra.run.smithi177.stdout:4608 punch from 0x2349659 to 0x2354869, (0xb210 bytes) 2024-05-08T03:20:33.955 INFO:teuthology.orchestra.run.smithi177.stdout:4609 read 0x5d50d84 thru 0x5d5db1b (0xcd98 bytes) 2024-05-08T03:20:33.955 INFO:teuthology.orchestra.run.smithi177.stdout:4611 read 0x555e9c0 thru 0x5566300 (0x7941 bytes) 2024-05-08T03:20:33.955 INFO:teuthology.orchestra.run.smithi177.stdout:4612 trunc from 0x93f24ea to 0xa4e9bdf 2024-05-08T03:20:33.960 INFO:teuthology.orchestra.run.smithi177.stdout:4613 punch from 0x6844ebc to 0x6848c9b, (0x3ddf bytes) 2024-05-08T03:20:33.965 INFO:teuthology.orchestra.run.smithi177.stdout:4614 write 0xab235aa thru 0xab27231 (0x3c88 bytes) 2024-05-08T03:20:33.977 INFO:teuthology.orchestra.run.smithi177.stdout:4615 trunc from 0xab27232 to 0x2d2af00 2024-05-08T03:20:33.996 INFO:teuthology.orchestra.run.smithi177.stdout:4616 writesame 0x5cf67b3 thru 0x5d0479a data_size 0x4(0xdfe8 bytes) 2024-05-08T03:20:33.996 INFO:teuthology.orchestra.run.smithi177.stdout:4618 read 0x9e7c07 thru 0x9ef2d4 (0x76ce bytes) 2024-05-08T03:20:34.006 INFO:teuthology.orchestra.run.smithi177.stdout:4619 trunc from 0x5d0479b to 0xe6e2296 2024-05-08T03:20:34.012 INFO:teuthology.orchestra.run.smithi177.stdout:4622 write 0x34edda7 thru 0x34f29c6 (0x4c20 bytes) 2024-05-08T03:20:34.015 INFO:teuthology.orchestra.run.smithi177.stdout:4623 write 0xd852509 thru 0xd85e4ae (0xbfa6 bytes) 2024-05-08T03:20:34.018 INFO:teuthology.orchestra.run.smithi177.stdout:4624 read 0x5d71310 thru 0x5d7f35f (0xe050 bytes) 2024-05-08T03:20:34.018 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:34.022 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:34.020+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:34.025 INFO:teuthology.orchestra.run.smithi177.stdout:4626 trunc from 0xe6e2296 to 0x89e1e93 2024-05-08T03:20:34.027 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.027 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.036 INFO:teuthology.orchestra.run.smithi177.stdout:4629 read 0x7989848 thru 0x798b71c (0x1ed5 bytes) 2024-05-08T03:20:34.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:34.044 INFO:teuthology.orchestra.run.smithi177.stdout:4630 writesame 0xb184082 thru 0xb188ad1 data_size 0x10(0x4a50 bytes) 2024-05-08T03:20:34.044 INFO:teuthology.orchestra.run.smithi177.stdout:4631 writesame 0x64a1d1e thru 0x64ad09b data_size 0x19(0xb37e bytes) 2024-05-08T03:20:34.047 INFO:teuthology.orchestra.run.smithi177.stdout:4632 writesame 0x70fdf7a thru 0x7101215 data_size 0x4(0x329c bytes) 2024-05-08T03:20:34.052 INFO:teuthology.orchestra.run.smithi177.stdout:4633 writesame 0x284510 thru 0x291d87 data_size 0xc(0xd878 bytes) 2024-05-08T03:20:34.055 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:34.055 INFO:teuthology.orchestra.run.smithi177.stdout:4636 write 0x43882ce thru 0x438d05d (0x4d90 bytes) 2024-05-08T03:20:34.058 INFO:teuthology.orchestra.run.smithi177.stdout:4637 trunc from 0xb188ad2 to 0xb10d046 2024-05-08T03:20:34.063 INFO:teuthology.orchestra.run.smithi177.stdout:4640 read 0x73284a8 thru 0x7331cfa (0x9853 bytes) 2024-05-08T03:20:34.063 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:34.063 INFO:teuthology.orchestra.run.smithi177.stdout:4642 write 0x5d64788 thru 0x5d6613f (0x19b8 bytes) 2024-05-08T03:20:34.065 INFO:teuthology.orchestra.run.smithi177.stdout:4643 read 0x4da8603 thru 0x4dae5fb (0x5ff9 bytes) 2024-05-08T03:20:34.073 INFO:teuthology.orchestra.run.smithi177.stdout:4644 trunc from 0xb10d046 to 0x4442638 2024-05-08T03:20:34.088 INFO:teuthology.orchestra.run.smithi177.stdout:4645 write 0x13f305 thru 0x14a701 (0xb3fd bytes) 2024-05-08T03:20:34.099 INFO:teuthology.orchestra.run.smithi177.stdout:4647 trunc from 0x4442638 to 0xc81f5b0 2024-05-08T03:20:34.105 INFO:teuthology.orchestra.run.smithi177.stdout:4650 read 0x38d7ff9 thru 0x38dcff7 (0x4fff bytes) 2024-05-08T03:20:34.111 INFO:teuthology.orchestra.run.smithi177.stdout:4651 trunc from 0xc81f5b0 to 0x771be4f 2024-05-08T03:20:34.118 INFO:teuthology.orchestra.run.smithi177.stdout:4652 read 0x6c719f7 thru 0x6c7e4fc (0xcb06 bytes) 2024-05-08T03:20:34.118 INFO:teuthology.orchestra.run.smithi177.stdout:4654 read 0x1baa81f thru 0x1bb920f (0xe9f1 bytes) 2024-05-08T03:20:34.118 INFO:teuthology.orchestra.run.smithi177.stdout:4656 read 0x3960a0 thru 0x3a1fc2 (0xbf23 bytes) 2024-05-08T03:20:34.118 INFO:teuthology.orchestra.run.smithi177.stdout:4658 punch from 0x3b94d26 to 0x3b9dc95, (0x8f6f bytes) 2024-05-08T03:20:34.118 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:34.118 INFO:teuthology.orchestra.run.smithi177.stdout:4661 read 0x4d560e3 thru 0x4d62862 (0xc780 bytes) 2024-05-08T03:20:34.122 INFO:teuthology.orchestra.run.smithi177.stdout:4662 trunc from 0x771be4f to 0x3c60da7 2024-05-08T03:20:34.122 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:34.121+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:34.133 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:34.146 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.147 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.151 INFO:teuthology.orchestra.run.smithi177.stdout:4664 write 0xdbf6a67 thru 0xdc04218 (0xd7b2 bytes) 2024-05-08T03:20:34.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:34.157 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:34.157 INFO:teuthology.orchestra.run.smithi177.stdout:4666 punch from 0x7828498 to 0x7835f2c, (0xda94 bytes) 2024-05-08T03:20:34.157 INFO:teuthology.orchestra.run.smithi177.stdout:4667 writesame 0x3a37dc thru 0x3a4bdc data_size 0x3(0x1401 bytes) 2024-05-08T03:20:34.162 INFO:teuthology.orchestra.run.smithi177.stdout:4669 trunc from 0xdc04219 to 0xa3b2d1a 2024-05-08T03:20:34.190 INFO:teuthology.orchestra.run.smithi177.stderr:marked in osd.4. 2024-05-08T03:20:34.194 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:20:34.210 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-05-08T03:20:34.252 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:34.250+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:34.267 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.267 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.275 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:34.325 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:34.323+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:34.380 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.381 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:34.425 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:34.424+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:34.495 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.495 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.504 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:34.527 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:34.525+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:34.608 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.608 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.617 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:34.628 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:34.626+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:34.726 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.727 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.729 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:34.727+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:34.735 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:34.773 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:20:34.830 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:34.828+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:34.880 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.881 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:34.889 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:34.933 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:34.931+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:34.995 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:34.995 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.004 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:35.045 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:35.043+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:35.133 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:35.131+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:35.149 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.149 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.158 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:35.234 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:35.232+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:35.264 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.264 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.273 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:35.334 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:35.333+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:35.379 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.379 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.388 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:35.472 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:35.471+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:35.535 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.535 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.536 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:35.535+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:35.544 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:35.637 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:35.636+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:35.689 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.690 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.698 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:35.740 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:35.739+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:35.852 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.852 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:35.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:35.871 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:35.838+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:35.899 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:35.935 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:35.935 INFO:teuthology.orchestra.run.smithi177.stdout:4674 write 0xebbfdc thru 0xec066f (0x4694 bytes) 2024-05-08T03:20:35.940 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:35.939+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:35.941 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:35.941 INFO:teuthology.orchestra.run.smithi177.stdout:4676 punch from 0x87318d to 0x879387, (0x61fa bytes) 2024-05-08T03:20:35.941 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:35.941 INFO:teuthology.orchestra.run.smithi177.stdout:4679 writesame 0x8adf4bf thru 0x8aec385 data_size 0x5(0xcec7 bytes) 2024-05-08T03:20:35.948 INFO:teuthology.orchestra.run.smithi177.stdout:4680 read 0x3d6642e thru 0x3d6ba04 (0x55d7 bytes) 2024-05-08T03:20:35.948 INFO:teuthology.orchestra.run.smithi177.stdout:4682 writesame 0x4438b43 thru 0x4442df6 data_size 0x59(0xa2b4 bytes) 2024-05-08T03:20:35.959 INFO:teuthology.orchestra.run.smithi177.stdout:4683 write 0x5033b66 thru 0x50401de (0xc679 bytes) 2024-05-08T03:20:35.983 INFO:teuthology.orchestra.run.smithi177.stdout:4684 writesame 0xee547bc thru 0xee5bb6d data_size 0x2(0x73b2 bytes) 2024-05-08T03:20:35.984 INFO:teuthology.orchestra.run.smithi177.stdout:4685 write 0xea7064e thru 0xea77c00 (0x75b3 bytes) 2024-05-08T03:20:35.990 INFO:teuthology.orchestra.run.smithi177.stdout:4687 write 0xe57311b thru 0xe582710 (0xf5f6 bytes) 2024-05-08T03:20:35.997 INFO:teuthology.orchestra.run.smithi177.stdout:4689 write 0x615c995 thru 0x61675db (0xac47 bytes) 2024-05-08T03:20:35.998 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:35.998 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:36.002 INFO:teuthology.orchestra.run.smithi177.stdout:4693 write 0x5a0461e thru 0x5a11f11 (0xd8f4 bytes) 2024-05-08T03:20:36.006 INFO:teuthology.orchestra.run.smithi177.stdout:4694 write 0xaed749a thru 0xaed7d5a (0x8c1 bytes) 2024-05-08T03:20:36.010 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:36.011 INFO:teuthology.orchestra.run.smithi177.stdout:4696 write 0x37cb19c thru 0x37d01a6 (0x500b bytes) 2024-05-08T03:20:36.014 INFO:teuthology.orchestra.run.smithi177.stdout:4697 write 0xa93251c thru 0xa935289 (0x2d6e bytes) 2024-05-08T03:20:36.019 INFO:teuthology.orchestra.run.smithi177.stdout:4698 write 0x3919242 thru 0x39227fb (0x95ba bytes) 2024-05-08T03:20:36.027 INFO:teuthology.orchestra.run.smithi177.stdout:4699 writesame 0x6cfb063 thru 0x6d03fb7 data_size 0x9(0x8f55 bytes) 2024-05-08T03:20:36.034 INFO:teuthology.orchestra.run.smithi177.stdout:4703 punch from 0xee59705 to 0xee5bb6e, (0x2469 bytes) 2024-05-08T03:20:36.034 INFO:teuthology.orchestra.run.smithi177.stdout:4704 punch from 0x1edb222 to 0x1ee071b, (0x54f9 bytes) 2024-05-08T03:20:36.034 INFO:teuthology.orchestra.run.smithi177.stdout:4705 read 0x783b8b thru 0x78624f (0x26c5 bytes) 2024-05-08T03:20:36.034 INFO:teuthology.orchestra.run.smithi177.stdout:4706 write 0x624d124 thru 0x6251fec (0x4ec9 bytes) 2024-05-08T03:20:36.038 INFO:teuthology.orchestra.run.smithi177.stdout:4708 punch from 0xb6b881b to 0xb6bb82a, (0x300f bytes) 2024-05-08T03:20:36.038 INFO:teuthology.orchestra.run.smithi177.stdout:4709 write 0x544c5f8 thru 0x5459368 (0xcd71 bytes) 2024-05-08T03:20:36.041 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:36.040+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:36.042 INFO:teuthology.orchestra.run.smithi177.stdout:4710 write 0x3f8d3c0 thru 0x3f96ae3 (0x9724 bytes) 2024-05-08T03:20:36.046 INFO:teuthology.orchestra.run.smithi177.stdout:4712 writesame 0x7fe291c thru 0x7fe9388 data_size 0x1(0x6a6d bytes) 2024-05-08T03:20:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:36.077 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:4713 trunc from 0xee5bb6e to 0x888c823 2024-05-08T03:20:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:36.098 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:4715 read 0x9e6c83 thru 0x9f52ee (0xe66c bytes) 2024-05-08T03:20:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:4718 read 0x3d971eb thru 0x3d9f1a7 (0x7fbd bytes) 2024-05-08T03:20:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:4720 write 0x1bec906 thru 0x1bee551 (0x1c4c bytes) 2024-05-08T03:20:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:4721 read 0x42c1547 thru 0x42cde16 (0xc8d0 bytes) 2024-05-08T03:20:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:4722 read 0x10bda3 thru 0x111a07 (0x5c65 bytes) 2024-05-08T03:20:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:4725 write 0x4a3321e thru 0x4a37189 (0x3f6c bytes) 2024-05-08T03:20:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:4726 read 0x605e1c2 thru 0x6063917 (0x5756 bytes) 2024-05-08T03:20:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:4728 write 0x7fbaab9 thru 0x7fc5963 (0xaeab bytes) 2024-05-08T03:20:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:4729 read 0x1ce948b thru 0x1cf098d (0x7503 bytes) 2024-05-08T03:20:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:4731 read 0xb0a85c thru 0xb19cc9 (0xf46e bytes) 2024-05-08T03:20:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:36.120 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:4732 writesame 0xbae5fae thru 0xbaf28cf data_size 0x13(0xc922 bytes) 2024-05-08T03:20:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:4734 write 0xb2a1470 thru 0xb2adab5 (0xc646 bytes) 2024-05-08T03:20:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:4735 write 0xa7b1095 thru 0xa7bffca (0xef36 bytes) 2024-05-08T03:20:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:4737 read 0x98f6f31 thru 0x98fb985 (0x4a55 bytes) 2024-05-08T03:20:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:4738 write 0x1ff4425 thru 0x1ff787b (0x3457 bytes) 2024-05-08T03:20:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:4739 write 0x4ea80c3 thru 0x4eb302a (0xaf68 bytes) 2024-05-08T03:20:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:4742 read 0x3bdf180 thru 0x3be69f3 (0x7874 bytes) 2024-05-08T03:20:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:4744 writesame 0x7130189 thru 0x7138ec9 data_size 0x1(0x8d41 bytes) 2024-05-08T03:20:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:36.141 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:36.142 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:36.140+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:36.151 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:36.151 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:36.151 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:36.164 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:36.166 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:36.166 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:4746 punch from 0x6db8898 to 0x6dbc03f, (0x37a7 bytes) 2024-05-08T03:20:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:4747 read 0x4380d60 thru 0x438d160 (0xc401 bytes) 2024-05-08T03:20:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:4748 read 0x45eddc2 thru 0x45fcd7d (0xefbc bytes) 2024-05-08T03:20:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:4749 trunc from 0xbaf28d0 to 0xc5847d3 2024-05-08T03:20:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:36.191 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:4753 write 0xe268eba thru 0xe26e7ac (0x58f3 bytes) 2024-05-08T03:20:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:4754 punch from 0x3a5459a to 0x3a58e8a, (0x48f0 bytes) 2024-05-08T03:20:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:4755 write 0xa1e43c5 thru 0xa1f1e4a (0xda86 bytes) 2024-05-08T03:20:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:4758 write 0xdfb7c09 thru 0xdfba5b2 (0x29aa bytes) 2024-05-08T03:20:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:36.213 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:4759 trunc from 0xe26e7ad to 0xa16fe7a 2024-05-08T03:20:36.218 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:36.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:36.236 INFO:teuthology.orchestra.run.smithi177.stdout:4761 read 0x5e179c7 thru 0x5e1a95c (0x2f96 bytes) 2024-05-08T03:20:36.236 INFO:teuthology.orchestra.run.smithi177.stdout:4762 write 0x9f2d7b1 thru 0x9f3bafa (0xe34a bytes) 2024-05-08T03:20:36.239 INFO:teuthology.orchestra.run.smithi177.stdout:4763 punch from 0xd1289f to 0xd1f83e, (0xcf9f bytes) 2024-05-08T03:20:36.239 INFO:teuthology.orchestra.run.smithi177.stdout:4764 read 0x36233da thru 0x362ec79 (0xb8a0 bytes) 2024-05-08T03:20:36.239 INFO:teuthology.orchestra.run.smithi177.stdout:4765 punch from 0x200d606 to 0x200e0cc, (0xac6 bytes) 2024-05-08T03:20:36.239 INFO:teuthology.orchestra.run.smithi177.stdout:4766 write 0x74d1046 thru 0x74d2081 (0x103c bytes) 2024-05-08T03:20:36.242 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:36.241+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:36.246 INFO:teuthology.orchestra.run.smithi177.stdout:4768 trunc from 0xa16fe7a to 0xd486f05 2024-05-08T03:20:36.258 INFO:teuthology.orchestra.run.smithi177.stdout:4770 trunc from 0xd486f05 to 0x8f2266c 2024-05-08T03:20:36.270 INFO:teuthology.orchestra.run.smithi177.stdout:4771 write 0x533c3ff thru 0x533e2cb (0x1ecd bytes) 2024-05-08T03:20:36.274 INFO:teuthology.orchestra.run.smithi177.stdout:4772 writesame 0x8c3b791 thru 0x8c3fec0 data_size 0x2(0x4730 bytes) 2024-05-08T03:20:36.286 INFO:teuthology.orchestra.run.smithi177.stdout:4774 write 0x576ff1f thru 0x577882b (0x890d bytes) 2024-05-08T03:20:36.298 INFO:teuthology.orchestra.run.smithi177.stdout:4777 writesame 0xcec6d6f thru 0xceccc99 data_size 0x3(0x5f2b bytes) 2024-05-08T03:20:36.299 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:36.299 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:36.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:36.299 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:36.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.299 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.300618288999999, 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.300550825000002, 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.305 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:36.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:36.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:36.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:36.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:36.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.311 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.240716493000001, 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:36.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:36.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.174208479999997, 2024-05-08T03:20:36.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:36.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:36.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:36.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.147670834000003, 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.323 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:36.328 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.119759066, 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0411788729999998, 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:20:36.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:20:36.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:20:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71203 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 561152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0365713620000001, 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075437740000000003, 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71203 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260426+0000", 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260428+0000", 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260435+0000", 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3260000000000001e-06 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260440+0000", 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8189999999999999e-06 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335711+0000", 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075271494999999994 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335784+0000", 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2786000000000004e-05 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335864+0000", 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9684999999999993e-05 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0365514969999996, 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.341 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:36.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:20:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:20:36.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0365308619999993, 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:20:36.345 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:36.349 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0365086110000004, 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:36.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0364881799999992, 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:20:36.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:36.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0364679540000008, 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.358 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:20:36.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.362 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0364471599999998, 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:20:36.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:20:36.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 8.0364211460000003, 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.370 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.8564642259999999, 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:20:36.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:20:36.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:20:36.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:20:36.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.8564451220000002, 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.375 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:20:36.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.8558835479999995, 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:20:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.383 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.8506696869999999, 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:36.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:20:36.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:20:36.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:20:36.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 7.8505023459999999, 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:36.389 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:36.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:20:36.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:20:36.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:20:36.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.394 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:36.395 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:36.395 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:36.396 INFO:teuthology.orchestra.run.smithi177.stdout:4778 trunc from 0xceccc9a to 0xa9f2f3 2024-05-08T03:20:36.396 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:36.394+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:36.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:36.398 INFO:teuthology.orchestra.run.smithi177.stdout:4779 write 0x59fa311 thru 0x5a01e25 (0x7b15 bytes) 2024-05-08T03:20:36.398 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.398 INFO:teuthology.orchestra.run.smithi177.stdout:4781 writesame 0xe429c35 thru 0xe429cb3 data_size 0x1(0x7f bytes) 2024-05-08T03:20:36.398 INFO:teuthology.orchestra.run.smithi177.stdout:4782 trunc from 0xe429cb4 to 0x7f6b771 2024-05-08T03:20:36.398 INFO:teuthology.orchestra.run.smithi177.stdout:4783 writesame 0xd5c2ed2 thru 0xd5ca769 data_size 0x22(0x7898 bytes) 2024-05-08T03:20:36.398 INFO:teuthology.orchestra.run.smithi177.stdout:4785 read 0x9f52e17 thru 0x9f554e2 (0x26cc bytes) 2024-05-08T03:20:36.405 INFO:teuthology.orchestra.run.smithi177.stdout:4786 trunc from 0xd5ca76a to 0x196efec 2024-05-08T03:20:36.421 INFO:teuthology.orchestra.run.smithi177.stdout:4789 read 0x27001a thru 0x27e48d (0xe474 bytes) 2024-05-08T03:20:36.422 INFO:teuthology.orchestra.run.smithi177.stdout:4790 read 0x1189434 thru 0x1191dda (0x89a7 bytes) 2024-05-08T03:20:36.422 INFO:teuthology.orchestra.run.smithi177.stdout:4791 punch from 0xf332f9 to 0xf41747, (0xe44e bytes) 2024-05-08T03:20:36.443 INFO:teuthology.orchestra.run.smithi177.stdout:4793 write 0xb0744e3 thru 0xb075aa9 (0x15c7 bytes) 2024-05-08T03:20:36.456 INFO:teuthology.orchestra.run.smithi177.stdout:4794 read 0x31e0f9c thru 0x31e7460 (0x64c5 bytes) 2024-05-08T03:20:36.466 INFO:teuthology.orchestra.run.smithi177.stdout:4795 trunc from 0xb075aaa to 0x3178375 2024-05-08T03:20:36.479 INFO:teuthology.orchestra.run.smithi177.stdout:4798 punch from 0x17900ca to 0x17961cb, (0x6101 bytes) 2024-05-08T03:20:36.479 INFO:teuthology.orchestra.run.smithi177.stdout:4799 read 0x8c5b69 thru 0x8cc4a4 (0x693c bytes) 2024-05-08T03:20:36.479 INFO:teuthology.orchestra.run.smithi177.stdout:4800 read 0x1413f05 thru 0x141dc53 (0x9d4f bytes) 2024-05-08T03:20:36.497 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:36.495+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:36.498 INFO:teuthology.orchestra.run.smithi177.stdout:4801 write 0xe34fdc4 thru 0xe352e2c (0x3069 bytes) 2024-05-08T03:20:36.502 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:36.502 INFO:teuthology.orchestra.run.smithi177.stdout:4803 writesame 0xdcdd14c thru 0xdceb8c4 data_size 0x1(0xe779 bytes) 2024-05-08T03:20:36.510 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:36.510 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:36.510 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:36.510 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:36.520 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:36.534 INFO:teuthology.orchestra.run.smithi177.stdout:4804 punch from 0x4a352b to 0x4a8578, (0x504d bytes) 2024-05-08T03:20:36.534 INFO:teuthology.orchestra.run.smithi177.stdout:4805 trunc from 0xe352e2d to 0xe4e5541 2024-05-08T03:20:36.543 INFO:teuthology.orchestra.run.smithi177.stdout:4806 read 0xd05dfe2 thru 0xd0646b4 (0x66d3 bytes) 2024-05-08T03:20:36.543 INFO:teuthology.orchestra.run.smithi177.stdout:4807 write 0x4fc1ffb thru 0x4fc474f (0x2755 bytes) 2024-05-08T03:20:36.550 INFO:teuthology.orchestra.run.smithi177.stdout:4811 read 0x909d1ce thru 0x90a9108 (0xbf3b bytes) 2024-05-08T03:20:36.550 INFO:teuthology.orchestra.run.smithi177.stdout:4814 read 0x4378c9a thru 0x437b3b9 (0x2720 bytes) 2024-05-08T03:20:36.550 INFO:teuthology.orchestra.run.smithi177.stdout:4815 read 0x5a8ffc4 thru 0x5a918d0 (0x190d bytes) 2024-05-08T03:20:36.550 INFO:teuthology.orchestra.run.smithi177.stdout:4816 read 0xb3308d7 thru 0xb33aae4 (0xa20e bytes) 2024-05-08T03:20:36.550 INFO:teuthology.orchestra.run.smithi177.stdout:4817 read 0xf0124d thru 0xf1010f (0xeec3 bytes) 2024-05-08T03:20:36.551 INFO:teuthology.orchestra.run.smithi177.stdout:4818 punch from 0x97507c1 to 0x975a409, (0x9c48 bytes) 2024-05-08T03:20:36.551 INFO:teuthology.orchestra.run.smithi177.stdout:4819 write 0x7308904 thru 0x730f0ee (0x67eb bytes) 2024-05-08T03:20:36.553 INFO:teuthology.orchestra.run.smithi177.stdout:4820 clone 36 order 18 su 65536 sc 10 2024-05-08T03:20:36.598 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:36.596+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:36.629 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:36.629 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:36.638 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:36.699 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:36.698+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:813 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.882888730000005, 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085811959999999993, 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.753 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 813 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867657+0000", 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2200000000000004e-07 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867661+0000", 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7409999999999998e-06 2024-05-08T03:20:36.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867670+0000", 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5790000000000004e-06 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867676+0000", 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869335+0000", 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001659081 2024-05-08T03:20:36.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869432+0000", 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6645999999999996e-05 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869888+0000", 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000456043 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874147+0000", 2024-05-08T03:20:36.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.004258604 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876171+0000", 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020245319999999999 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876202+0000", 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0939999999999999e-05 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876238+0000", 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5839e-05 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:815 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.882546551999994, 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083276170000000007, 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.759 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 815 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.867999+0000", 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868000+0000", 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3900000000000003e-07 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868005+0000", 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3800000000000002e-06 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868017+0000", 2024-05-08T03:20:36.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1826e-05 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868025+0000", 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9219999999999997e-06 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870323+0000", 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002297964 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870371+0000", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8001999999999999e-05 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.870681+0000", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000310358 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874912+0000", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042305390000000002 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876272+0000", 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013598849999999999 2024-05-08T03:20:36.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876301+0000", 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9232999999999999e-05 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876326+0000", 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5568999999999999e-05 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:816 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.764 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.882507720000007, 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083942019999999999, 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 816 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868038+0000", 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2099999999999997e-07 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868054+0000", 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5845000000000002e-05 2024-05-08T03:20:36.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868063+0000", 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2809999999999997e-06 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868069+0000", 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9179999999999999e-06 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871071+0000", 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0030017889999999999 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871147+0000", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5940999999999997e-05 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871414+0000", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00026663199999999999 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874919+0000", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035051230000000002 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876365+0000", 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0014458940000000001 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876392+0000", 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7801e-05 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876432+0000", 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9456999999999998e-05 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:818 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.882461687000003, 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010676412, 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 818 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868084+0000", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4799999999999998e-07 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868088+0000", 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5319999999999998e-06 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868095+0000", 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6909999999999997e-06 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.868101+0000", 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5099999999999998e-06 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871832+0000", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003731233 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.871908+0000", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5990999999999998e-05 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872235+0000", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00032720600000000002 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874925+0000", 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026892869999999998 2024-05-08T03:20:36.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878688+0000", 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0037635329999999999 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878726+0000", 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7994999999999998e-05 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878760+0000", 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3886e-05 2024-05-08T03:20:36.774 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:820 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.881444494999997, 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0097765379999999995, 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 820 2024-05-08T03:20:36.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869101+0000", 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869102+0000", 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999999e-07 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869106+0000", 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3950000000000003e-06 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869118+0000", 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1651000000000001e-05 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869126+0000", 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8199999999999997e-06 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872639+0000", 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035138220000000002 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.872714+0000", 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4549999999999996e-05 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873057+0000", 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034304399999999998 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874929+0000", 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018725300000000001 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878809+0000", 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003879824 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878840+0000", 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0874999999999998e-05 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878877+0000", 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7245e-05 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:821 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.881408243999999, 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098243870000000004, 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.781 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 821 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869137+0000", 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869138+0000", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869141+0000", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0510000000000001e-06 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869149+0000", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5529999999999993e-06 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869155+0000", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6960000000000002e-06 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873442+0000", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0042873829999999996 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873529+0000", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6446999999999994e-05 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.873873+0000", 2024-05-08T03:20:36.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000344323 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876133+0000", 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0022604460000000002 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878910+0000", 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002776274 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878938+0000", 2024-05-08T03:20:36.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.777e-05 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878962+0000", 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3986999999999999e-05 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:823 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.881380945000004, 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0098967529999999995, 2024-05-08T03:20:36.786 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 823 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869164+0000", 2024-05-08T03:20:36.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869165+0000", 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9799999999999999e-07 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869168+0000", 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9519999999999999e-06 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869181+0000", 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2938e-05 2024-05-08T03:20:36.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869186+0000", 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4179999999999996e-06 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874341+0000", 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0051550750000000003 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874412+0000", 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0852999999999994e-05 2024-05-08T03:20:36.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874662+0000", 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00024959100000000002 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876335+0000", 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016736959999999999 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.878996+0000", 2024-05-08T03:20:36.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0026611130000000001 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879024+0000", 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7294999999999999e-05 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879061+0000", 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7423999999999999e-05 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.791 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:824 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.881351252000002, 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015557371, 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 824 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.792 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869194+0000", 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869195+0000", 2024-05-08T03:20:36.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3000000000000001e-07 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869198+0000", 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0149999999999999e-06 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869205+0000", 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3379999999999999e-06 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869212+0000", 2024-05-08T03:20:36.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5939999999999996e-06 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.874979+0000", 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0057660250000000001 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875030+0000", 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1965999999999997e-05 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875270+0000", 2024-05-08T03:20:36.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00023905000000000001 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.876930+0000", 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016604009999999999 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884697+0000", 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0077671420000000003 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884726+0000", 2024-05-08T03:20:36.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8804999999999999e-05 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884751+0000", 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5605e-05 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:825 3.1a 3:5e68eb20:::rbd_object_map.11821677beed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.881325736999997, 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015640248999999998, 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.797 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 825 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869220+0000", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.51e-07 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869225+0000", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9930000000000003e-06 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869233+0000", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.869238+0000", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.464e-06 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875627+0000", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063895430000000001 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875689+0000", 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1966999999999998e-05 2024-05-08T03:20:36.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.875898+0000", 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020897600000000001 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.877289+0000", 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0013908309999999999 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884793+0000", 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075038070000000004 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884829+0000", 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6307999999999999e-05 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.884860+0000", 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0685999999999997e-05 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:840 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:head [truncate 4194304] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.802 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.871211571000003, 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0082557910000000002, 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 840 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879334+0000", 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6700000000000003e-07 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879337+0000", 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:36.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879342+0000", 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4809999999999997e-06 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.879346+0000", 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7469999999999999e-06 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880155+0000", 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00080957700000000002 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880205+0000", 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9209000000000003e-05 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.880307+0000", 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000102361 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.881324+0000", 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010165980000000001 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887535+0000", 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0062112110000000003 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887564+0000", 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9561999999999999e-05 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:27.887590+0000", 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5157999999999999e-05 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1343 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5459968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618872726000006, 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076625249999999999, 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1343 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.808 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131673+0000", 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4700000000000002e-07 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131675+0000", 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8199999999999999e-06 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131681+0000", 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.229e-06 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131685+0000", 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2320000000000003e-06 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139241+0000", 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0075560649999999998 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139295+0000", 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3510000000000001e-05 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139335+0000", 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0321999999999998e-05 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1344 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5468160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618854395, 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078306850000000004, 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1344 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131691+0000", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4499999999999998e-07 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131693+0000", 2024-05-08T03:20:36.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8339999999999999e-06 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131699+0000", 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3010000000000003e-06 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131704+0000", 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3109999999999999e-06 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139365+0000", 2024-05-08T03:20:36.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0076612370000000004 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139447+0000", 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2293999999999995e-05 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139522+0000", 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4363000000000006e-05 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.815 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1345 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5476352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618829934000004, 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079162069999999998, 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1345 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:36.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131715+0000", 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131716+0000", 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2600000000000002e-07 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131724+0000", 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131731+0000", 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6590000000000001e-06 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131736+0000", 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.578e-06 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139547+0000", 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0078112060000000002 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139589+0000", 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2657999999999997e-05 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139632+0000", 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2159000000000001e-05 2024-05-08T03:20:36.819 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1346 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5484544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618803591000002, 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080747700000000002, 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.820 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1346 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:36.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131742+0000", 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7599999999999998e-07 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131744+0000", 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.778e-06 2024-05-08T03:20:36.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131750+0000", 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131754+0000", 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4100000000000001e-06 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139662+0000", 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079080739999999993 2024-05-08T03:20:36.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139742+0000", 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9653999999999993e-05 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139817+0000", 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4827999999999996e-05 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1347 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5492736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.824 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618786200000002, 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081548669999999997, 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.825 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1347 2024-05-08T03:20:36.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.826 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.826 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.826 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:36.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131759+0000", 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131760+0000", 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131761+0000", 2024-05-08T03:20:36.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7010000000000001e-06 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131767+0000", 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131771+0000", 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139831+0000", 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0080600859999999993 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139874+0000", 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2997000000000002e-05 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139914+0000", 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9944000000000002e-05 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1348 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5500928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:36.829 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618768919000004, 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083201400000000002, 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1348 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:36.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131776+0000", 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131777+0000", 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2800000000000003e-07 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131783+0000", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5939999999999999e-06 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131789+0000", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1260000000000003e-06 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131798+0000", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.139944+0000", 2024-05-08T03:20:36.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0081463609999999995 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140024+0000", 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9636999999999997e-05 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140097+0000", 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2744999999999996e-05 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.834 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1349 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5509120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.834 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:36.834 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618741482000004, 2024-05-08T03:20:36.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001939999999997, 2024-05-08T03:20:36.835 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.835 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.835 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.835 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.835 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1349 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131804+0000", 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4099999999999999e-07 2024-05-08T03:20:36.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131806+0000", 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131812+0000", 2024-05-08T03:20:36.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8039999999999998e-06 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131816+0000", 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2490000000000003e-06 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140120+0000", 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0083040200000000005 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140164+0000", 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3457000000000001e-05 2024-05-08T03:20:36.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140204+0000", 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0262999999999997e-05 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1350 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5517312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618723523, 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.008564248, 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.840 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1350 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.841 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131822+0000", 2024-05-08T03:20:36.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-07 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131824+0000", 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.733e-06 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131829+0000", 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5249999999999996e-06 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131834+0000", 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2560000000000004e-06 2024-05-08T03:20:36.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140234+0000", 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0084001739999999998 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140315+0000", 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1057000000000007e-05 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140386+0000", 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1175999999999998e-05 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.844 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1351 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5525504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618706437, 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086476150000000009, 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1351 2024-05-08T03:20:36.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131839+0000", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6600000000000002e-07 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131841+0000", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7209999999999999e-06 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131846+0000", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2190000000000003e-06 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131850+0000", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2579999999999997e-06 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140405+0000", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0085541430000000002 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140446+0000", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1770999999999997e-05 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140487+0000", 2024-05-08T03:20:36.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0136999999999999e-05 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:1352 3.1a 3:5a9031ef:::rbd_data.11821677beed.0000000000000005:5 [read 5533696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 68.618689453000002, 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0088024279999999993, 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:36.849 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 1352 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:36.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131856+0000", 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.22e-07 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131858+0000", 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.671e-06 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131863+0000", 2024-05-08T03:20:36.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5709999999999997e-06 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.131868+0000", 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5090000000000004e-06 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140508+0000", 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0086399700000000003 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140587+0000", 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9367000000000001e-05 2024-05-08T03:20:36.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:28.140658+0000", 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1018000000000003e-05 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:36.853 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:36.853 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:20:36.855 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:36.856 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:36.853+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:36.956 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:36.954+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:36.967 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:36.967 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:36.967 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:36.967 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:36.976 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:37.057 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:37.055+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:37.089 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:37.089 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:37.089 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:37.089 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:37.089 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:37.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:37.158 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:37.156+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:37.215 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 61.004901977000003, 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.216 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:20:37.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:20:37.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:20:37.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:20:37.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.08522146, 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.221 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:20:37.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085197389000001, 2024-05-08T03:20:37.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085171666000001, 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:20:37.229 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:37.233 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085139949000002, 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:20:37.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085114845, 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:20:37.237 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:37.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085090039999997, 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.241 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085061496000002, 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.245 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.246 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085038974, 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.249 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.250 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:20:37.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.085008633999998, 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.253 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:20:37.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.084987452999997, 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.257 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:37.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:37.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.260 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.272345516999998, 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.262 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:20:37.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:20:37.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.083670796, 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:20:37.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:37.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.061467452999999, 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:37.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:20:37.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.059771335000001, 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:20:37.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.279 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.040066547000002, 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:37.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.040042587000002, 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:20:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:20:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.289 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.999451041999997, 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:20:37.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.293 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.986677145000002, 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 17.725938210999999, 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:20:37.301 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.302 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.302 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:37.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:37.304 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:37.301+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:37.404 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:37.402+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:37.421 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:37.421 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:37.421 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:37.421 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:37.430 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:37.504 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:37.503+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:37.545 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:37.545 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:37.545 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:37.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:37.605 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:37.604+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 65.799533828999998, 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.676 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:20:37.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.682 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457592637000005, 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:37.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:20:37.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:20:37.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.686 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457577237999999, 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:37.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:20:37.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.690 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457545596000003, 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457532263999994, 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:20:37.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457522470000001, 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:20:37.698 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.702 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457513057, 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:37.706 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457503536000004, 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:20:37.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:20:37.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.710 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457494194999995, 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:20:37.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457484980000004, 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:20:37.714 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:20:37.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457475618000004, 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:20:37.718 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:37.722 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457466186000005, 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:37.726 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457456769999993, 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457447751999993, 2024-05-08T03:20:37.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:20:37.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457438709000002, 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.734 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:37.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:20:37.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457426416000004, 2024-05-08T03:20:37.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.742 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457417148999994, 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:37.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:20:37.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457403291000006, 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.747 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:20:37.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.457393998000001, 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.755 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 61.464379864999998, 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:37.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:37.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:37.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:37.760 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:37.761 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:37.762 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:37.760+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:37.776 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:20:37.862 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:37.860+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:37.886 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:37.886 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:37.886 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:37.887 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:37.897 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:37.963 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:37.961+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:38.014 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:38.014 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:38.014 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:38.014 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:38.014 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:38.023 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:38.064 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:38.063+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:38.165 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:38.163+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.970960238000004, 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.173 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:38.177 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.970950787999996, 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.970938051999994, 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.185 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.970927476, 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:38.186 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:38.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.190 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.970918088999994, 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:38.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.970908909000002, 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.198 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.740083874000007, 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:38.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:38.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:38.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.740051636999993, 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.740038307000006, 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:38.211 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.740023332999996, 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.215 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.740012023999995, 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:38.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.219 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.739994354999993, 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:38.223 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.739979156999993, 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:38.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.227 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.739965063, 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.228 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.965011089999997, 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.042117288, 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:38.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:38.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.0421038, 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.241 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:38.245 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.042074819, 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:38.249 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.042048213000001, 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:38.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.609299952000001, 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.256 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:38.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:38.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.262 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:38.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:38.263 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:38.307 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:38.305+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:38.367 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:38.365+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:38.417 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "MOSDPGPushReply(3.1a 85/82 [PushReplyOp(3:5ead3c65:::rbd_data.118227bb2f9.0000000000000001:13)])", 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:38.416231+0000", 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00061149800000000003, 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00062716400000000004, 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:38.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416231+0000", 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416231+0000", 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416231+0000", 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8400000000000004e-07 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416236+0000", 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3019999999999996e-06 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416247+0000", 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0973e-05 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416253+0000", 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4999999999999996e-06 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:38.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:38.416306+0000", 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2320000000000001e-05 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:20:38.421 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:38.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:38.433 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:38.434 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:38.458 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:38.458 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:38.468 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:38.466+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:38.474 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:38.474 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:38.491 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:38.492 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:38.509 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:38.510 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:38.510 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:38.528 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:38.528 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:38.543 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:38.543 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:38.546 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:38.546 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:38.549 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:38.549 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:38.553 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:38.594 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:38.592+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:38.669 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:38.668+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:38.701 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:38.702 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:38.711 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:38.770 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:38.768+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:38.869 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:38.869 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:38.870 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:38.869+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:38.878 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:38.971 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:38.969+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:39.028 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:39.028 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:39.038 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:39.082 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:39.081+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:39.151 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:39.151 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:39.160 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:39.173 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:39.171+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:39.211 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 4] out_osds: [7, 0, 5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:39.416 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:39.416 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T03:20:39.416 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:20:39.418 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:39.418 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:39.418 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:39.419 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:39.416+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:39.519 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:39.517+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:39.571 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:39.571 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:39.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:39.619 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:39.618+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:39.720 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:39.719+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:39.730 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:39.731 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:39.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:39.768 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 86, 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:20:38.782137+0000", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:20:08.032979+0000", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:20:34.185270+0000", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5804032, 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "nodeep-scrub", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:20:39.769 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 5, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "luminous", 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:20:39.770 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:39.771 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.772 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:39.773 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:39.774 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:39.775 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.6500000953674316, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.6500000953674316, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.70999997854232788, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:39.776 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:39.777 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:39.778 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:39.779 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:39.780 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:20:39.781 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 6.9000000953674316, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 6.9000000953674316, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:39.782 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:39.783 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:39.784 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "84", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 38, 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 84, 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:39.785 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:39.786 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:39.787 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 2.4200000762939453, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 2.8599998950958252, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.70999997854232788, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.7200000286102295, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2.0299999713897705, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:39.788 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 40, 2024-05-08T03:20:39.789 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.790 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.792 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:20:39.793 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:39.794 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 85, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.799 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 85, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.800 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.801 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.802 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.803 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:20:39.804 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.805 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 85, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.806 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.807 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:20:39.808 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:39.809 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:39.810 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 85, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.811 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:20:39.812 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 0, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 45, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 61, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.816 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.817 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.818 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.819 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1643429991", 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1643429991", 2024-05-08T03:20:39.820 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1643429991", 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1643429991", 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "exists" 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.821 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 54, 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.822 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.823 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.824 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:20:39.825 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.826 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.827 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.828 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.829 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:39.830 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:20:39.831 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:39.832 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:39.833 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.834 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.835 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 61 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.836 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:20:39.837 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [ 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "1.5", 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: 2, 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: 0 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.d", 2024-05-08T03:20:39.838 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: 6 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [ 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.6", 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 1, 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.12", 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 3, 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.17", 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 3, 2024-05-08T03:20:39.840 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [ 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.3", 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: 1, 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: 6 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.841 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.e", 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: 1 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.1b", 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: 1 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.1c", 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:39.842 INFO:teuthology.orchestra.run.smithi177.stdout: 1 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:20:39.843 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [], 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [], 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:39.844 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:39.845 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:20:39.847 INFO:tasks.thrashosds.thrasher:j is {'epoch': 86, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:20:38.782137+0000', 'last_up_change': '2024-05-08T03:20:08.032979+0000', 'last_in_change': '2024-05-08T03:20:34.185270+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 38, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.4200000762939453, 'score_stable': 2.859999895095825, 'optimal_score': 0.7099999785423279, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 2.0299999713897705, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 45, 'down_at': 61, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1643429991}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1643429991}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1643429991}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1643429991}]}, 'public_addr': '172.21.15.188:6817/1643429991', 'cluster_addr': '172.21.15.188:6819/1643429991', 'heartbeat_back_addr': '172.21.15.188:6823/1643429991', 'heartbeat_front_addr': '172.21.15.188:6821/1643429991', 'state': ['exists']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 61}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.5', 'osds': [2, 0]}, {'pgid': '3.d', 'osds': [3, 6]}], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '3.12', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.17', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.3', 'osds': [1, 6]}, {'pgid': '3.e', 'osds': [1]}, {'pgid': '3.1b', 'osds': [1]}, {'pgid': '3.1c', 'osds': [1]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:20:39.847 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:20:39.848 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:39.844+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:39.851 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:39.851 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:39.860 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:39.947 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:39.945+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:40.005 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:40.005 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:40.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:40.047 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:40.045+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:40.148 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:40.147+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:40.160 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:40.160 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:40.169 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:40.206 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:40.206 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:20:40.218 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-05-08T03:20:40.187531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1560639471,"num_objects":1827,"num_object_clones":413,"num_object_copies":3654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":812,"num_objects_misplaced":500,"num_objects_unfound":0,"num_objects_dirty":1827,"num_whiteouts":709,"num_read":138962,"num_read_kb":987032,"num_write":15253,"num_write_kb":620267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":171540588,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11932,"ondisk_log_size":11932,"up":82,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":1127576,"kb_used_data":882832,"kb_used_omap":7,"kb_used_meta":244664,"kb_avail":470731624,"statfs":{"total":483183820800,"available":482029182976,"internally_reserved":0,"allocated":904019968,"data_stored":1303952046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7972,"internal_metadata":250536156},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25379337,"num_objects":80,"num_object_clones":9,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":812,"num_objects_misplaced":500,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":20,"num_read":5574,"num_read_kb":60543,"num_write":1263,"num_write_kb":10141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":9985116,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000635"},"pg_stats":[{"pgid":"3.1f","version":"80'230","reported_seq":2438,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.775683+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:20:35.775683+0000","last_peered":"2024-05-08T03:20:35.775683+0000","last_clean":"2024-05-08T03:20:35.775683+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:20:35.775683+0000","last_undegraded":"2024-05-08T03:20:35.775683+0000","last_fullsized":"2024-05-08T03:20:35.775683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:20:10.271273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":56233586,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":2465,"num_read_kb":29049,"num_write":292,"num_write_kb":15738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"80'189","reported_seq":1601,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.781566+0000","last_change":"2024-05-08T03:19:22.850001+0000","last_active":"2024-05-08T03:20:35.781566+0000","last_peered":"2024-05-08T03:20:35.781566+0000","last_clean":"2024-05-08T03:20:35.781566+0000","last_became_active":"2024-05-08T03:19:20.773128+0000","last_became_peered":"2024-05-08T03:19:20.773128+0000","last_unstale":"2024-05-08T03:20:35.781566+0000","last_undegraded":"2024-05-08T03:20:35.781566+0000","last_fullsized":"2024-05-08T03:20:35.781566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:53.116595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":23291636,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":1587,"num_read_kb":4228,"num_write":190,"num_write_kb":2731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"80'347","reported_seq":5726,"reported_epoch":83,"state":"active+recovery_wait+remapped","last_fresh":"2024-05-08T03:20:35.920555+0000","last_change":"2024-05-08T03:20:35.920716+0000","last_active":"2024-05-08T03:20:35.920555+0000","last_peered":"2024-05-08T03:20:35.920555+0000","last_clean":"2024-05-08T03:20:34.076629+0000","last_became_active":"2024-05-08T03:20:35.919999+0000","last_became_peered":"2024-05-08T03:20:35.919999+0000","last_unstale":"2024-05-08T03:20:35.920555+0000","last_undegraded":"2024-05-08T03:20:35.920555+0000","last_fullsized":"2024-05-08T03:20:35.920555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":69913496,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":88,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":6077,"num_read_kb":41466,"num_write":434,"num_write_kb":13383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":278351,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[{"shards":"1,6","objects":54}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"83'774","reported_seq":3514,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:35.959134+0000","last_change":"2024-05-08T03:20:35.929609+0000","last_active":"2024-05-08T03:20:35.959134+0000","last_peered":"2024-05-08T03:20:35.959134+0000","last_clean":"2024-05-08T03:20:33.980855+0000","last_became_active":"2024-05-08T03:20:35.928761+0000","last_became_peered":"2024-05-08T03:20:35.928761+0000","last_unstale":"2024-05-08T03:20:35.959134+0000","last_undegraded":"2024-05-08T03:20:35.779566+0000","last_fullsized":"2024-05-08T03:20:35.774620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":57776167,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":83,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":5313,"num_read_kb":51668,"num_write":1153,"num_write_kb":50900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1994536,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","6"],"object_location_counts":[{"shards":"1","objects":1},{"shards":"1,6","objects":51}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"80'350","reported_seq":3010,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:35.927241+0000","last_change":"2024-05-08T03:20:35.927502+0000","last_active":"2024-05-08T03:20:35.927241+0000","last_peered":"2024-05-08T03:20:35.927241+0000","last_clean":"2024-05-08T03:20:33.402771+0000","last_became_active":"2024-05-08T03:20:35.926657+0000","last_became_peered":"2024-05-08T03:20:35.926657+0000","last_unstale":"2024-05-08T03:20:35.927241+0000","last_undegraded":"2024-05-08T03:20:35.781502+0000","last_fullsized":"2024-05-08T03:20:35.777885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":37793902,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":3962,"num_read_kb":19720,"num_write":443,"num_write_kb":13267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2376803,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":52}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"83'445","reported_seq":3724,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:36.186258+0000","last_change":"2024-05-08T03:20:35.924340+0000","last_active":"2024-05-08T03:20:36.186258+0000","last_peered":"2024-05-08T03:20:36.186258+0000","last_clean":"2024-05-08T03:20:34.150485+0000","last_became_active":"2024-05-08T03:20:35.923533+0000","last_became_peered":"2024-05-08T03:20:35.923533+0000","last_unstale":"2024-05-08T03:20:36.186258+0000","last_undegraded":"2024-05-08T03:20:35.780387+0000","last_fullsized":"2024-05-08T03:20:35.775296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":54647095,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":4121,"num_read_kb":38908,"num_write":612,"num_write_kb":43033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":91773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1","objects":1},{"shards":"1,2","objects":52}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"83'238","reported_seq":2160,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.082655+0000","last_change":"2024-05-08T03:19:22.779129+0000","last_active":"2024-05-08T03:20:36.082655+0000","last_peered":"2024-05-08T03:20:36.082655+0000","last_clean":"2024-05-08T03:20:36.082655+0000","last_became_active":"2024-05-08T03:19:20.775300+0000","last_became_peered":"2024-05-08T03:19:20.775300+0000","last_unstale":"2024-05-08T03:20:36.082655+0000","last_undegraded":"2024-05-08T03:20:36.082655+0000","last_fullsized":"2024-05-08T03:20:36.082655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:20.221991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":43953464,"num_objects":56,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":2012,"num_read_kb":10106,"num_write":249,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"80'393","reported_seq":7706,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.781526+0000","last_change":"2024-05-08T03:19:53.957566+0000","last_active":"2024-05-08T03:20:35.781526+0000","last_peered":"2024-05-08T03:20:35.781526+0000","last_clean":"2024-05-08T03:20:35.781526+0000","last_became_active":"2024-05-08T03:19:53.957072+0000","last_became_peered":"2024-05-08T03:19:53.957072+0000","last_unstale":"2024-05-08T03:20:35.781526+0000","last_undegraded":"2024-05-08T03:20:35.781526+0000","last_fullsized":"2024-05-08T03:20:35.781526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:38:42.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":75024481,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":25,"num_read":9079,"num_read_kb":56082,"num_write":412,"num_write_kb":13990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":6126489,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"80'268","reported_seq":3806,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.775639+0000","last_change":"2024-05-08T03:20:06.164893+0000","last_active":"2024-05-08T03:20:35.775639+0000","last_peered":"2024-05-08T03:20:35.775639+0000","last_clean":"2024-05-08T03:20:35.775639+0000","last_became_active":"2024-05-08T03:20:06.035685+0000","last_became_peered":"2024-05-08T03:20:06.035685+0000","last_unstale":"2024-05-08T03:20:35.775639+0000","last_undegraded":"2024-05-08T03:20:35.775639+0000","last_fullsized":"2024-05-08T03:20:35.775639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:33.490858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":25983109,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":4288,"num_read_kb":20740,"num_write":292,"num_write_kb":3319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":192270,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"83'176","reported_seq":2622,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.088894+0000","last_change":"2024-05-08T03:19:57.081242+0000","last_active":"2024-05-08T03:20:36.088894+0000","last_peered":"2024-05-08T03:20:36.088894+0000","last_clean":"2024-05-08T03:20:36.088894+0000","last_became_active":"2024-05-08T03:19:56.989658+0000","last_became_peered":"2024-05-08T03:19:56.989658+0000","last_unstale":"2024-05-08T03:20:36.088894+0000","last_undegraded":"2024-05-08T03:20:36.088894+0000","last_fullsized":"2024-05-08T03:20:36.088894+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:41:48.611344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":90960933,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":22,"num_read":2262,"num_read_kb":60781,"num_write":174,"num_write_kb":44775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":8790475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"80'218","reported_seq":5610,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:35.919281+0000","last_change":"2024-05-08T03:20:35.919490+0000","last_active":"2024-05-08T03:20:35.919281+0000","last_peered":"2024-05-08T03:20:35.919281+0000","last_clean":"2024-05-08T03:20:33.980793+0000","last_became_active":"2024-05-08T03:20:35.918582+0000","last_became_peered":"2024-05-08T03:20:35.918582+0000","last_unstale":"2024-05-08T03:20:35.919281+0000","last_undegraded":"2024-05-08T03:20:35.784742+0000","last_fullsized":"2024-05-08T03:20:35.780703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":49534636,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":5493,"num_read_kb":32966,"num_write":238,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":268258,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3],"avail_no_missing":["3","6"],"object_location_counts":[{"shards":"3,6","objects":61}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"83'617","reported_seq":3243,"reported_epoch":83,"state":"active+recovery_wait+remapped","last_fresh":"2024-05-08T03:20:36.099810+0000","last_change":"2024-05-08T03:20:35.925653+0000","last_active":"2024-05-08T03:20:36.099810+0000","last_peered":"2024-05-08T03:20:36.099810+0000","last_clean":"2024-05-08T03:20:34.126622+0000","last_became_active":"2024-05-08T03:20:35.924891+0000","last_became_peered":"2024-05-08T03:20:35.924891+0000","last_unstale":"2024-05-08T03:20:36.099810+0000","last_undegraded":"2024-05-08T03:20:36.099810+0000","last_fullsized":"2024-05-08T03:20:36.099810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":28316462,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":90,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":23,"num_read":4881,"num_read_kb":16496,"num_write":840,"num_write_kb":6214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":135734,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[{"shards":"2,6","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.782647+0000","last_change":"2024-05-08T03:20:03.022786+0000","last_active":"2024-05-08T03:20:35.782647+0000","last_peered":"2024-05-08T03:20:35.782647+0000","last_clean":"2024-05-08T03:20:35.782647+0000","last_became_active":"2024-05-08T03:20:03.022530+0000","last_became_peered":"2024-05-08T03:20:03.022530+0000","last_unstale":"2024-05-08T03:20:35.782647+0000","last_undegraded":"2024-05-08T03:20:35.782647+0000","last_fullsized":"2024-05-08T03:20:35.782647+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:59.431605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"83'261","reported_seq":2758,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.125362+0000","last_change":"2024-05-08T03:19:57.064707+0000","last_active":"2024-05-08T03:20:36.125362+0000","last_peered":"2024-05-08T03:20:36.125362+0000","last_clean":"2024-05-08T03:20:36.125362+0000","last_became_active":"2024-05-08T03:19:56.989832+0000","last_became_peered":"2024-05-08T03:19:56.989832+0000","last_unstale":"2024-05-08T03:20:36.125362+0000","last_undegraded":"2024-05-08T03:20:36.125362+0000","last_fullsized":"2024-05-08T03:20:36.125362+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:58.961127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":15838918,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":25,"num_read":2566,"num_read_kb":9702,"num_write":295,"num_write_kb":3971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":208697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.781483+0000","last_change":"2024-05-08T03:20:03.023919+0000","last_active":"2024-05-08T03:20:35.781483+0000","last_peered":"2024-05-08T03:20:35.781483+0000","last_clean":"2024-05-08T03:20:35.781483+0000","last_became_active":"2024-05-08T03:20:03.023611+0000","last_became_peered":"2024-05-08T03:20:03.023611+0000","last_unstale":"2024-05-08T03:20:35.781483+0000","last_undegraded":"2024-05-08T03:20:35.781483+0000","last_fullsized":"2024-05-08T03:20:35.781483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:04:35.966897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'579","reported_seq":3397,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.045465+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:20:36.045465+0000","last_peered":"2024-05-08T03:20:36.045465+0000","last_clean":"2024-05-08T03:20:36.045465+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:20:36.045465+0000","last_undegraded":"2024-05-08T03:20:36.045465+0000","last_fullsized":"2024-05-08T03:20:36.045465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:30:02.295244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":25865790,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":20,"num_read":5222,"num_read_kb":29768,"num_write":799,"num_write_kb":17283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.914106+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:20:35.914106+0000","last_peered":"2024-05-08T03:20:35.914106+0000","last_clean":"2024-05-08T03:20:35.914106+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:20:35.914106+0000","last_undegraded":"2024-05-08T03:20:35.914106+0000","last_fullsized":"2024-05-08T03:20:35.914106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"83'207","reported_seq":3937,"reported_epoch":83,"state":"active+undersized+remapped","last_fresh":"2024-05-08T03:20:36.138634+0000","last_change":"2024-05-08T03:20:36.096361+0000","last_active":"2024-05-08T03:20:36.138634+0000","last_peered":"2024-05-08T03:20:36.138634+0000","last_clean":"2024-05-08T03:20:34.157381+0000","last_became_active":"2024-05-08T03:20:35.918337+0000","last_became_peered":"2024-05-08T03:20:35.918337+0000","last_unstale":"2024-05-08T03:20:36.138634+0000","last_undegraded":"2024-05-08T03:20:36.138634+0000","last_fullsized":"2024-05-08T03:20:35.780338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":34026008,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":3434,"num_read_kb":26935,"num_write":215,"num_write_kb":8450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":6689531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3],"avail_no_missing":["3","1","4"],"object_location_counts":[{"shards":"3","objects":53}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.780632+0000","last_change":"2024-05-08T03:19:56.989000+0000","last_active":"2024-05-08T03:20:35.780632+0000","last_peered":"2024-05-08T03:20:35.780632+0000","last_clean":"2024-05-08T03:20:35.780632+0000","last_became_active":"2024-05-08T03:19:56.988667+0000","last_became_peered":"2024-05-08T03:19:56.988667+0000","last_unstale":"2024-05-08T03:20:35.780632+0000","last_undegraded":"2024-05-08T03:20:35.780632+0000","last_fullsized":"2024-05-08T03:20:35.780632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:33.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"83'779","reported_seq":3099,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:36.000805+0000","last_change":"2024-05-08T03:20:35.929627+0000","last_active":"2024-05-08T03:20:36.000805+0000","last_peered":"2024-05-08T03:20:36.000805+0000","last_clean":"2024-05-08T03:20:33.519431+0000","last_became_active":"2024-05-08T03:20:35.928752+0000","last_became_peered":"2024-05-08T03:20:35.928752+0000","last_unstale":"2024-05-08T03:20:36.000805+0000","last_undegraded":"2024-05-08T03:20:35.774050+0000","last_fullsized":"2024-05-08T03:20:35.771831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":40010219,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":94,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":4977,"num_read_kb":29634,"num_write":1244,"num_write_kb":31930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":30566047,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2],"avail_no_missing":["2","6"],"object_location_counts":[{"shards":"2","objects":1},{"shards":"2,6","objects":62}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"83'179","reported_seq":2559,"reported_epoch":83,"state":"active+recovering+remapped","last_fresh":"2024-05-08T03:20:36.201668+0000","last_change":"2024-05-08T03:20:36.096534+0000","last_active":"2024-05-08T03:20:36.201668+0000","last_peered":"2024-05-08T03:20:36.201668+0000","last_clean":"2024-05-08T03:20:33.375939+0000","last_became_active":"2024-05-08T03:20:35.918232+0000","last_became_peered":"2024-05-08T03:20:35.918232+0000","last_unstale":"2024-05-08T03:20:36.201668+0000","last_undegraded":"2024-05-08T03:20:36.201668+0000","last_fullsized":"2024-05-08T03:20:36.201668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":19754920,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":2026,"num_read_kb":12395,"num_write":216,"num_write_kb":18705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":5303202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":37}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.773261+0000","last_change":"2024-05-08T03:19:53.957635+0000","last_active":"2024-05-08T03:20:35.773261+0000","last_peered":"2024-05-08T03:20:35.773261+0000","last_clean":"2024-05-08T03:20:35.773261+0000","last_became_active":"2024-05-08T03:19:53.957252+0000","last_became_peered":"2024-05-08T03:19:53.957252+0000","last_unstale":"2024-05-08T03:20:35.773261+0000","last_undegraded":"2024-05-08T03:20:35.773261+0000","last_fullsized":"2024-05-08T03:20:35.773261+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:27.307894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.781141+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:20:35.781141+0000","last_peered":"2024-05-08T03:20:35.781141+0000","last_clean":"2024-05-08T03:20:35.781141+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:20:35.781141+0000","last_undegraded":"2024-05-08T03:20:35.781141+0000","last_fullsized":"2024-05-08T03:20:35.781141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"83'240","reported_seq":4433,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.773209+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:20:35.773209+0000","last_peered":"2024-05-08T03:20:35.773209+0000","last_clean":"2024-05-08T03:20:35.773209+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:20:35.773209+0000","last_undegraded":"2024-05-08T03:20:35.773209+0000","last_fullsized":"2024-05-08T03:20:35.773209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:16.955261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":34333380,"num_objects":71,"num_object_clones":20,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":4639,"num_read_kb":15715,"num_write":217,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.918813+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:20:35.918813+0000","last_peered":"2024-05-08T03:20:35.918813+0000","last_clean":"2024-05-08T03:20:35.918813+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:20:35.918813+0000","last_undegraded":"2024-05-08T03:20:35.918813+0000","last_fullsized":"2024-05-08T03:20:35.918813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"80'599","reported_seq":6206,"reported_epoch":83,"state":"active+recovery_wait+remapped","last_fresh":"2024-05-08T03:20:35.927608+0000","last_change":"2024-05-08T03:20:35.927608+0000","last_active":"2024-05-08T03:20:35.927608+0000","last_peered":"2024-05-08T03:20:35.927608+0000","last_clean":"2024-05-08T03:20:33.996327+0000","last_became_active":"2024-05-08T03:20:35.926673+0000","last_became_peered":"2024-05-08T03:20:35.926673+0000","last_unstale":"2024-05-08T03:20:35.927608+0000","last_undegraded":"2024-05-08T03:20:35.927608+0000","last_fullsized":"2024-05-08T03:20:35.927608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":59839713,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":84,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":21,"num_read":8592,"num_read_kb":39171,"num_write":821,"num_write_kb":20172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":481376,"num_keys_recovered":30,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[{"shards":"1,6","objects":61}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.772146+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:20:35.772146+0000","last_peered":"2024-05-08T03:20:35.772146+0000","last_clean":"2024-05-08T03:20:35.772146+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:20:35.772146+0000","last_undegraded":"2024-05-08T03:20:35.772146+0000","last_fullsized":"2024-05-08T03:20:35.772146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"83'589","reported_seq":7571,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.780456+0000","last_change":"2024-05-08T03:19:59.997039+0000","last_active":"2024-05-08T03:20:35.780456+0000","last_peered":"2024-05-08T03:20:35.780456+0000","last_clean":"2024-05-08T03:20:35.780456+0000","last_became_active":"2024-05-08T03:19:59.996378+0000","last_became_peered":"2024-05-08T03:19:59.996378+0000","last_unstale":"2024-05-08T03:20:35.780456+0000","last_undegraded":"2024-05-08T03:20:35.780456+0000","last_fullsized":"2024-05-08T03:20:35.780456+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:08:15.829327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":49221631,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":25,"num_read":8761,"num_read_kb":47516,"num_write":831,"num_write_kb":15421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":6220729,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.773015+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:20:35.773015+0000","last_peered":"2024-05-08T03:20:35.773015+0000","last_clean":"2024-05-08T03:20:35.773015+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:20:35.773015+0000","last_undegraded":"2024-05-08T03:20:35.773015+0000","last_fullsized":"2024-05-08T03:20:35.773015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"80'369","reported_seq":3215,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.781616+0000","last_change":"2024-05-08T03:19:22.850234+0000","last_active":"2024-05-08T03:20:35.781616+0000","last_peered":"2024-05-08T03:20:35.781616+0000","last_clean":"2024-05-08T03:20:35.781616+0000","last_became_active":"2024-05-08T03:19:20.773661+0000","last_became_peered":"2024-05-08T03:19:20.773661+0000","last_unstale":"2024-05-08T03:20:35.781616+0000","last_undegraded":"2024-05-08T03:20:35.781616+0000","last_fullsized":"2024-05-08T03:20:35.781616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:50.725383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":21757691,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5339,"num_read_kb":12403,"num_write":369,"num_write_kb":5848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"83'221","reported_seq":2824,"reported_epoch":83,"state":"active+recovery_wait+remapped","last_fresh":"2024-05-08T03:20:35.940248+0000","last_change":"2024-05-08T03:20:35.919840+0000","last_active":"2024-05-08T03:20:35.940248+0000","last_peered":"2024-05-08T03:20:35.940248+0000","last_clean":"2024-05-08T03:20:33.317747+0000","last_became_active":"2024-05-08T03:20:35.918306+0000","last_became_peered":"2024-05-08T03:20:35.918306+0000","last_unstale":"2024-05-08T03:20:35.940248+0000","last_undegraded":"2024-05-08T03:20:35.940248+0000","last_fullsized":"2024-05-08T03:20:35.940248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":81242578,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":88,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":2459,"num_read_kb":17925,"num_write":250,"num_write_kb":16853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":5551754,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"83'347","reported_seq":1388,"reported_epoch":83,"state":"active+recovery_wait+remapped","last_fresh":"2024-05-08T03:20:36.194844+0000","last_change":"2024-05-08T03:20:35.922188+0000","last_active":"2024-05-08T03:20:36.194844+0000","last_peered":"2024-05-08T03:20:36.194844+0000","last_clean":"2024-05-08T03:20:34.150832+0000","last_became_active":"2024-05-08T03:20:35.920411+0000","last_became_peered":"2024-05-08T03:20:35.920411+0000","last_unstale":"2024-05-08T03:20:36.194844+0000","last_undegraded":"2024-05-08T03:20:36.194844+0000","last_fullsized":"2024-05-08T03:20:36.194844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":25357900,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":62,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":2111,"num_read_kb":9247,"num_write":374,"num_write_kb":5213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":24576,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[{"shards":"1,6","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"80'279","reported_seq":4284,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:35.932452+0000","last_change":"2024-05-08T03:20:35.932692+0000","last_active":"2024-05-08T03:20:35.932452+0000","last_peered":"2024-05-08T03:20:35.932452+0000","last_clean":"2024-05-08T03:20:33.519511+0000","last_became_active":"2024-05-08T03:20:35.931836+0000","last_became_peered":"2024-05-08T03:20:35.931836+0000","last_unstale":"2024-05-08T03:20:35.932452+0000","last_undegraded":"2024-05-08T03:20:35.782364+0000","last_fullsized":"2024-05-08T03:20:35.780816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":62518925,"num_objects":69,"num_object_clones":16,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":25,"num_read":4319,"num_read_kb":50397,"num_write":342,"num_write_kb":17231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2233399,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":69}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"83'458","reported_seq":4011,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:35.780579+0000","last_change":"2024-05-08T03:19:59.995470+0000","last_active":"2024-05-08T03:20:35.780579+0000","last_peered":"2024-05-08T03:20:35.780579+0000","last_clean":"2024-05-08T03:20:35.780579+0000","last_became_active":"2024-05-08T03:19:59.994538+0000","last_became_peered":"2024-05-08T03:19:59.994538+0000","last_unstale":"2024-05-08T03:20:35.780579+0000","last_undegraded":"2024-05-08T03:20:35.780579+0000","last_fullsized":"2024-05-08T03:20:35.780579+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:14.044311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":80392713,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":5551,"num_read_kb":19223,"num_write":614,"num_write_kb":16669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":8235378,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"83'705","reported_seq":3512,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.271993+0000","last_change":"2024-05-08T03:19:39.181175+0000","last_active":"2024-05-08T03:20:36.271993+0000","last_peered":"2024-05-08T03:20:36.271993+0000","last_clean":"2024-05-08T03:20:36.271993+0000","last_became_active":"2024-05-08T03:19:39.090307+0000","last_became_peered":"2024-05-08T03:19:39.090307+0000","last_unstale":"2024-05-08T03:20:36.271993+0000","last_undegraded":"2024-05-08T03:20:36.271993+0000","last_fullsized":"2024-05-08T03:20:36.271993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:11.536872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":79413441,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":6615,"num_read_kb":68946,"num_write":1045,"num_write_kb":47909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":993485,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"83'283","reported_seq":3215,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:35.970074+0000","last_change":"2024-05-08T03:20:35.924497+0000","last_active":"2024-05-08T03:20:35.970074+0000","last_peered":"2024-05-08T03:20:35.970074+0000","last_clean":"2024-05-08T03:20:34.014914+0000","last_became_active":"2024-05-08T03:20:35.923934+0000","last_became_peered":"2024-05-08T03:20:35.923934+0000","last_unstale":"2024-05-08T03:20:35.970074+0000","last_undegraded":"2024-05-08T03:20:35.779546+0000","last_fullsized":"2024-05-08T03:20:35.775361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":70144250,"num_objects":75,"num_object_clones":20,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":26,"num_read":2816,"num_read_kb":48321,"num_write":340,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":31431070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1","objects":1},{"shards":"1,3","objects":74}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"83'452","reported_seq":2934,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:36.041966+0000","last_change":"2024-05-08T03:20:35.920455+0000","last_active":"2024-05-08T03:20:36.041966+0000","last_peered":"2024-05-08T03:20:36.041966+0000","last_clean":"2024-05-08T03:20:33.650048+0000","last_became_active":"2024-05-08T03:20:35.919285+0000","last_became_peered":"2024-05-08T03:20:35.919285+0000","last_unstale":"2024-05-08T03:20:36.041966+0000","last_undegraded":"2024-05-08T03:20:35.774977+0000","last_fullsized":"2024-05-08T03:20:35.773625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":24837754,"num_objects":63,"num_object_clones":15,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":92,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":25,"num_read":3259,"num_read_kb":13229,"num_write":648,"num_write_kb":12703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4608126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1","objects":1},{"shards":"1,3","objects":62}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"83'197","reported_seq":2599,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.133113+0000","last_change":"2024-05-08T03:20:03.024129+0000","last_active":"2024-05-08T03:20:36.133113+0000","last_peered":"2024-05-08T03:20:36.133113+0000","last_clean":"2024-05-08T03:20:36.133113+0000","last_became_active":"2024-05-08T03:20:03.023616+0000","last_became_peered":"2024-05-08T03:20:03.023616+0000","last_unstale":"2024-05-08T03:20:36.133113+0000","last_undegraded":"2024-05-08T03:20:36.133113+0000","last_fullsized":"2024-05-08T03:20:36.133113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:20.551930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":68562006,"num_objects":63,"num_object_clones":15,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":2216,"num_read_kb":33580,"num_write":215,"num_write_kb":17169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":10514324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"83'306","reported_seq":4776,"reported_epoch":83,"state":"active+recovery_wait+remapped","last_fresh":"2024-05-08T03:20:36.177561+0000","last_change":"2024-05-08T03:20:35.922115+0000","last_active":"2024-05-08T03:20:36.177561+0000","last_peered":"2024-05-08T03:20:36.177561+0000","last_clean":"2024-05-08T03:20:33.059427+0000","last_became_active":"2024-05-08T03:20:35.921397+0000","last_became_peered":"2024-05-08T03:20:35.921397+0000","last_unstale":"2024-05-08T03:20:36.177561+0000","last_undegraded":"2024-05-08T03:20:36.177561+0000","last_fullsized":"2024-05-08T03:20:36.177561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":65677138,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":85,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":21,"num_read":5961,"num_read_kb":60327,"num_write":306,"num_write_kb":53398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":14483473,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[{"shards":"3,6","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"83'230","reported_seq":1255,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T03:20:36.103672+0000","last_change":"2024-05-08T03:20:06.097542+0000","last_active":"2024-05-08T03:20:36.103672+0000","last_peered":"2024-05-08T03:20:36.103672+0000","last_clean":"2024-05-08T03:20:36.103672+0000","last_became_active":"2024-05-08T03:20:06.033352+0000","last_became_peered":"2024-05-08T03:20:06.033352+0000","last_unstale":"2024-05-08T03:20:36.103672+0000","last_undegraded":"2024-05-08T03:20:36.103672+0000","last_fullsized":"2024-05-08T03:20:36.103672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:44:58.441338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":56132505,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":2814,"num_read_kb":23361,"num_write":232,"num_write_kb":23323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"83'290","reported_seq":3711,"reported_epoch":83,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-05-08T03:20:36.202951+0000","last_change":"2024-05-08T03:20:35.927391+0000","last_active":"2024-05-08T03:20:36.202951+0000","last_peered":"2024-05-08T03:20:36.202951+0000","last_clean":"2024-05-08T03:20:34.089322+0000","last_became_active":"2024-05-08T03:20:35.926582+0000","last_became_peered":"2024-05-08T03:20:35.926582+0000","last_unstale":"2024-05-08T03:20:36.202951+0000","last_undegraded":"2024-05-08T03:20:35.773118+0000","last_fullsized":"2024-05-08T03:20:35.771826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":31692656,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":3629,"num_read_kb":36927,"num_write":392,"num_write_kb":22733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":13456776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":["2","6"],"object_location_counts":[{"shards":"2","objects":2},{"shards":"2,6","objects":67}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1560049103,"num_objects":1825,"num_object_clones":413,"num_object_copies":3650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":812,"num_objects_misplaced":500,"num_objects_unfound":0,"num_objects_dirty":1825,"num_whiteouts":709,"num_read":138846,"num_read_kb":986932,"num_write":15094,"num_write_kb":617851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1286,"num_bytes_recovered":170359852,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1009020928,"data_stored":1460733441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11815,"ondisk_log_size":11815,"up":64,"acting":54,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542164,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310516,"kb_used_data":239080,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94061324,"statfs":{"total":96636764160,"available":96318795776,"internally_reserved":0,"allocated":244817920,"data_stored":364151401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":0,"seq":60129542159,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38172,"kb_used_data":6984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333668,"statfs":{"total":96636764160,"available":96597676032,"internally_reserved":0,"allocated":7151616,"data_stored":7205313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305680,"kb_used_data":256448,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94066160,"statfs":{"total":96636764160,"available":96323747840,"internally_reserved":0,"allocated":262602752,"data_stored":380575493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161300,"kb_used_data":121800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94210540,"statfs":{"total":96636764160,"available":96471592960,"internally_reserved":0,"allocated":124723200,"data_stored":179712886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311908,"kb_used_data":258520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94059932,"statfs":{"total":96636764160,"available":96317370368,"internally_reserved":0,"allocated":264724480,"data_stored":372306953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542165,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":263643136,"data_stored":371685790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124145664,"data_stored":179629285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":261308416,"data_stored":379763362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10477568,"data_stored":14021323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":243843072,"data_stored":363695286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:20:40.220 INFO:tasks.thrashosds.thrasher:Setting 3.19 to [3, 4] 2024-05-08T03:20:40.220 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.19', '3', '4'] 2024-05-08T03:20:40.220 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.19 3 4 2024-05-08T03:20:40.259 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:40.258+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:40.273 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:40.273 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:40.282 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:40.350 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:40.348+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:40.388 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:40.388 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:40.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:40.450 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:40.449+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:40.543 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:40.543 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:40.550 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:40.551 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:40.550+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:40.552 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:40.652 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:40.650+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:40.658 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:40.658 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:40.667 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:40.672 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:40.672 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:40.689 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:40.689 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:40.706 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:40.706 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:40.722 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:40.722 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:40.739 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:40.740 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:40.740 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:40.753 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:40.751+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:40.758 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:40.758 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:40.770 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:20:40.770 INFO:teuthology.orchestra.run.smithi177.stderr:set 3.19 pg_upmap mapping to [3,4] 2024-05-08T03:20:40.777 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:40.777 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:40.783 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:40.783 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:40.815 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-05-08T03:20:40.819 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:40.819 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:40.819 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:40.819 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:40.819 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:40.828 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:40.855 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:40.854+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.078468270000002, 2024-05-08T03:20:40.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736527077999995, 2024-05-08T03:20:40.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:20:40.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:20:40.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.965 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736511679000003, 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:20:40.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:20:40.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.969 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736480037000007, 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736466704999998, 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.974 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:20:40.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736456911000005, 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.981 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736447498000004, 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736437976999994, 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.986 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:20:40.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:20:40.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.989 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736428635999999, 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.990 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:20:40.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.993 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736419420999994, 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:20:40.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:20:40.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736410058999994, 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:40.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736400626999995, 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:20:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736391210999997, 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736382192999997, 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.013 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736373150000006, 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.017 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736360856999994, 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.018 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:41.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:20:41.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:20:41.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736351589999998, 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.022 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:20:41.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736337731999996, 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.026 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:20:41.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:20:41.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 67.736328439000005, 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:41.030 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 64.743314306000002, 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:41.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:41.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:41.038 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:41.039 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:40.953+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:41.040 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:41.084 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:41.082+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:41.145 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:41.145 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:41.145 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:41.146 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:41.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:41.196 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:41.194+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:41.257 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:41.255+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:41.300 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:41.300 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:41.300 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:41.300 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:41.300 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:41.309 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:41.393 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:41.392+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:41.457 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.255333023999995, 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.255323574000002, 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.255310838, 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.255300262000006, 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.255290875, 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.255281694999994, 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024456659999998, 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024424422999999, 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024411092999998, 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024396119000002, 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024384810000001, 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024367140999999, 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024351942999999, 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.024337849000005, 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.249383875999996, 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 64.326490074000006, 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:41.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:41.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:41.532 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 64.326476585999998, 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:41.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:41.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.536 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 64.326447604999998, 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 64.326420999000007, 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:41.541 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:41.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.893672737999999, 2024-05-08T03:20:41.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:41.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:41.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:41.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:41.551 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:41.553 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:41.553 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:41.551+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:41.654 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:41.652+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:41.701 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:41.701 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:41.701 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:41.701 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:41.710 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:41.794 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:41.793+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:41.856 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:41.855+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:41.858 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:41.858 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:41.858 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:41.858 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:41.858 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:41.867 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:41.957 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:41.955+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:42.015 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 56.227631504000001, 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:42.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 52.080397355000002, 2024-05-08T03:20:42.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.890257188, 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:42.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:42.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.865516642000003, 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.034 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:42.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:42.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:42.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67863 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:42.039 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.775584154000001, 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069622329999999996, 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67863 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-07 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237344+0000", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237349+0000", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7339999999999996e-06 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237354+0000", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.605e-06 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306659+0000", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069304893000000006 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306767+0000", 2024-05-08T03:20:42.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000108385 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306963+0000", 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00019608200000000001 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67864 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.775565178000001, 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069780320000000007, 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.043 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67864 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237362+0000", 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0679999999999999e-06 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237369+0000", 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4269999999999996e-06 2024-05-08T03:20:42.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237373+0000", 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.296e-06 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306990+0000", 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069617100000000001 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307041+0000", 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0796e-05 2024-05-08T03:20:42.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307140+0000", 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9189000000000002e-05 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67865 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.775546573, 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070114813999999998, 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.047 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67865 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:42.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.96e-07 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237381+0000", 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237386+0000", 2024-05-08T03:20:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.48e-06 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237392+0000", 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5620000000000003e-06 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307171+0000", 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069779439999999998 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307237+0000", 2024-05-08T03:20:42.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5838000000000002e-05 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307493+0000", 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025623099999999999 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67866 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.775527274, 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070792377000000004, 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67866 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999999e-07 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237401+0000", 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7630000000000001e-06 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237407+0000", 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7169999999999996e-06 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237412+0000", 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4729999999999999e-06 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307524+0000", 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070111455000000003 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307595+0000", 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1178000000000002e-05 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308190+0000", 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000595431 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67867 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.775506885, 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072514764999999995, 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67867 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237419+0000", 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7899999999999999e-07 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237421+0000", 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9860000000000001e-06 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237426+0000", 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.614e-06 2024-05-08T03:20:42.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237431+0000", 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2680000000000002e-06 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308233+0000", 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070801136000000001 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308945+0000", 2024-05-08T03:20:42.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00071273699999999996 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309933+0000", 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0009876450000000001 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67868 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.775487716000001, 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072516667000000007, 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.059 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67868 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239091+0000", 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001654006 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239096+0000", 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.921e-06 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239107+0000", 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0781000000000001e-05 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239114+0000", 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.939e-06 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309950+0000", 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070835871999999994 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309954+0000", 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1479999999999999e-06 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67869 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773798133, 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070844969999999993, 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.063 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67869 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239128+0000", 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239130+0000", 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.655e-06 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239138+0000", 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2690000000000002e-06 2024-05-08T03:20:42.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239143+0000", 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6690000000000003e-06 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309966+0000", 2024-05-08T03:20:42.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070823102999999998 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309972+0000", 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6350000000000003e-06 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67870 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773774404999999, 2024-05-08T03:20:42.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070850033000000007, 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67870 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239153+0000", 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2189999999999998e-06 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239160+0000", 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7170000000000002e-06 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239166+0000", 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4140000000000002e-06 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309994+0000", 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070828156000000003 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310001+0000", 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0310000000000001e-06 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67871 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773752483999999, 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070908609999999997, 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.071 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67871 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:42.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239175+0000", 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239182+0000", 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5640000000000002e-06 2024-05-08T03:20:42.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239187+0000", 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9409999999999998e-06 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310071+0000", 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070883710000000003 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310081+0000", 2024-05-08T03:20:42.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0716999999999999e-05 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67876 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773642039, 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070745004, 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.075 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67876 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:42.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239286+0000", 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0329999999999998e-06 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239291+0000", 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.57e-06 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239296+0000", 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5789999999999997e-06 2024-05-08T03:20:42.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310022+0000", 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070725882000000004 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310028+0000", 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67877 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773623214000001, 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070752494999999999, 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67877 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239305+0000", 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239311+0000", 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239316+0000", 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.527e-06 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310048+0000", 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070732489999999995 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310054+0000", 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2269999999999998e-06 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67878 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:42.082 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773603346, 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070795459000000005, 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67878 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8799999999999998e-07 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239324+0000", 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:42.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239330+0000", 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8050000000000003e-06 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239335+0000", 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5029999999999999e-06 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310110+0000", 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070775530000000003 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310117+0000", 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2089999999999999e-06 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67879 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773584833999999, 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070805241000000005, 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.086 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67879 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:42.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239341+0000", 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239343+0000", 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9400000000000001e-06 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239349+0000", 2024-05-08T03:20:42.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.533e-06 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239354+0000", 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4549999999999997e-06 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310138+0000", 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070784480999999996 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310146+0000", 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:42.089 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67880 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.773335474, 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070583059000000004, 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.090 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67880 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2200000000000003e-07 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239593+0000", 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8059999999999999e-06 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239600+0000", 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.584e-06 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239605+0000", 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6699999999999999e-06 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310166+0000", 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070560237999999997 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310173+0000", 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0389999999999999e-06 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:80533 3.f 3:f6536522:::rbd_data.11828c8e13f1.000000000000002f:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.078422075000001, 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083900980999999999, 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 80533 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934504+0000", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8299999999999996e-07 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934509+0000", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8520000000000003e-06 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934516+0000", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.058e-06 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934522+0000", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2029999999999997e-06 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.935441+0000", 2024-05-08T03:20:42.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000918745 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.935580+0000", 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013952300000000001 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.018404+0000", 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.082823816999999994 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.097 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:80614 3.16 3:696d6acd:::rbd_data.11828c8e13f1.0000000000000030:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 13.072862444, 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.078694985999999995, 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 80614 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6699999999999999e-07 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940066+0000", 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8329999999999998e-06 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940073+0000", 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1840000000000002e-06 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940079+0000", 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7139999999999998e-06 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.943185+0000", 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0031060459999999999 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.943333+0000", 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014836 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.018758+0000", 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075424381999999998 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.101 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.102 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.102 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:42.103 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:42.104 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:42.102+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:42.204 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:42.202+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:42.249 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:42.249 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:42.249 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:42.249 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:42.258 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:42.342 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:42.341+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:42.403 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:42.404 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:42.404 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:42.404 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:42.404 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:42.406 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:42.404+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:42.412 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:42.507 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:42.506+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:42.559 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:42.559 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.457516114999997, 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:42.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:42.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:42.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.455795645000002, 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.564 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:42.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:42.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.455762204000003, 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.573 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.454336464999997, 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:42.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:42.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.577 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.454243851000001, 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:42.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.453733176999997, 2024-05-08T03:20:42.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.586 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.453705669000001, 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.587 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:42.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:42.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.591 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.453299238, 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:42.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:42.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:42.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:42.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.452843860000002, 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.596 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.600 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.452315613000003, 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:42.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:42.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451843115000003, 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451764982999997, 2024-05-08T03:20:42.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451689561000002, 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.614 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:42.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:42.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:42.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451612398000002, 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:42.618 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451534875, 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:42.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:42.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:42.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451507962999997, 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:42.627 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.631 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451486752999998, 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:42.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:42.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.451466973000002, 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.636 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:42.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:42.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.450912447, 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 34.450893434999998, 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:42.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:42.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.649 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:42.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:42.651 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:42.649+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:42.752 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:42.750+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:42.784 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:42.794 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:20:42.802 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:42.802 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:42.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.802 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:101149 3.19 3.241aa539 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:40.831122+0000", 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.9699923479999999, 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.970013805, 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 101149 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831122+0000", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831122+0000", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831122+0000", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7899999999999999e-07 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831125+0000", 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2800000000000002e-06 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831130+0000", 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3190000000000002e-06 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831135+0000", 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6489999999999999e-06 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831377+0000", 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00024225900000000001 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.831386+0000", 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7080000000000003e-06 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775589+0000", 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94420322099999998 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775602+0000", 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3369e-05 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:101157 3.19 3.39c7bdb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:40.832149+0000", 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.968965418, 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9690351349999999, 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 101157 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832149+0000", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832149+0000", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832149+0000", 2024-05-08T03:20:42.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832153+0000", 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6689999999999999e-06 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832160+0000", 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3989999999999998e-06 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832167+0000", 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3459999999999999e-06 2024-05-08T03:20:42.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832221+0000", 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3825000000000001e-05 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.832230+0000", 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.465e-06 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775651+0000", 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94342062999999998 2024-05-08T03:20:42.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775659+0000", 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9869999999999999e-06 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:101306 3.19 3.e95bb79 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:40.835823+0000", 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.9652911579999999, 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.965401532, 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 101306 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835823+0000", 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835823+0000", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835824+0000", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000002e-07 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835827+0000", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2380000000000002e-06 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835834+0000", 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2080000000000003e-06 2024-05-08T03:20:42.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835840+0000", 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9320000000000001e-06 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835898+0000", 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7583000000000001e-05 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835907+0000", 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4909999999999994e-06 2024-05-08T03:20:42.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775752+0000", 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93984502599999997 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775759+0000", 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5830000000000003e-06 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.815 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:101308 3.19 3.79047bf9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:40.835876+0000", 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.965238059, 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9653884339999999, 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 101308 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835876+0000", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835876+0000", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835877+0000", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835893+0000", 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6385e-05 2024-05-08T03:20:42.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835900+0000", 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4379999999999998e-06 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835906+0000", 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1229999999999999e-06 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835943+0000", 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7920000000000003e-05 2024-05-08T03:20:42.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.835951+0000", 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4279999999999999e-06 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775802+0000", 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985101800000004 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775806+0000", 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999997e-06 2024-05-08T03:20:42.819 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:101205 3.19 3.74f34db9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:40.833031+0000", 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.968083619, 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968272287, 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 101205 2024-05-08T03:20:42.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833031+0000", 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833031+0000", 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833031+0000", 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7899999999999998e-07 2024-05-08T03:20:42.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833035+0000", 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.574e-06 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833042+0000", 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0500000000000003e-06 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833048+0000", 2024-05-08T03:20:42.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7339999999999996e-06 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833147+0000", 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9234000000000006e-05 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833156+0000", 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8489999999999995e-06 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775682+0000", 2024-05-08T03:20:42.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94252628500000002 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775690+0000", 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2460000000000003e-06 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:101208 3.19 3.8fc8ed19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:40.833140+0000", 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 1.967974152, 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9682006699999999, 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:42.824 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 101208 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833140+0000", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833140+0000", 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:42.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833141+0000", 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9399999999999995e-07 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833143+0000", 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6579999999999998e-06 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833161+0000", 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7938999999999999e-05 2024-05-08T03:20:42.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833166+0000", 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2830000000000001e-06 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833394+0000", 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00022708099999999999 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:40.833401+0000", 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7219999999999999e-06 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775716+0000", 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94231427999999995 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:41.775724+0000", 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2600000000000005e-06 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 6 2024-05-08T03:20:42.828 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:42.830 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:42.873 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:42.871+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:42.911 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:42.912 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:42.929 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:42.930 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:42.950 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:42.951 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:42.953 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:42.951+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:42.969 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:42.969 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:42.985 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:42.985 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:42.987 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:42.987 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:42.988 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:42.995 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:43.006 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:43.006 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:43.025 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:43.025 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:43.028 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:43.028 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:43.054 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:43.053+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:43.113 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.113 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.122 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:43.154 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:43.153+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:43.244 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.244 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:43.255 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:43.254+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:43.277 INFO:teuthology.orchestra.run.smithi177.stdout:4821 read 0x5bcfa44 thru 0x5bdbbb7 (0xc174 bytes) 2024-05-08T03:20:43.278 INFO:teuthology.orchestra.run.smithi177.stdout:4822 punch from 0x22e5111 to 0x22e56a8, (0x597 bytes) 2024-05-08T03:20:43.282 INFO:teuthology.orchestra.run.smithi177.stdout:4826 read 0x6b84b57 thru 0x6b937ce (0xec78 bytes) 2024-05-08T03:20:43.282 INFO:teuthology.orchestra.run.smithi177.stdout:4827 write 0x27803e0 thru 0x278c6c6 (0xc2e7 bytes) 2024-05-08T03:20:43.286 INFO:teuthology.orchestra.run.smithi177.stdout:4829 write 0x99a29c3 thru 0x99ade2e (0xb46c bytes) 2024-05-08T03:20:43.294 INFO:teuthology.orchestra.run.smithi177.stdout:4830 trunc from 0xe4e5541 to 0xa3e7c66 2024-05-08T03:20:43.309 INFO:teuthology.orchestra.run.smithi177.stdout:4831 read 0x5072266 thru 0x5074a4f (0x27ea bytes) 2024-05-08T03:20:43.310 INFO:teuthology.orchestra.run.smithi177.stdout:4834 writesame 0x7dce86b thru 0x7dd7130 data_size 0xe(0x88c6 bytes) 2024-05-08T03:20:43.314 INFO:teuthology.orchestra.run.smithi177.stdout:4835 writesame 0x6d9c53d thru 0x6daaf76 data_size 0x7(0xea3a bytes) 2024-05-08T03:20:43.322 INFO:teuthology.orchestra.run.smithi177.stdout:4836 write 0x3c04771 thru 0x3c0a10a (0x599a bytes) 2024-05-08T03:20:43.332 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.342 INFO:teuthology.orchestra.run.smithi177.stdout:4838 trunc from 0xa3e7c66 to 0x1c3e7dc 2024-05-08T03:20:43.356 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:43.355+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:43.376 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.376 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.382 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.382 INFO:teuthology.orchestra.run.smithi177.stdout:4840 punch from 0x14140a2 to 0x1417d13, (0x3c71 bytes) 2024-05-08T03:20:43.382 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.382 INFO:teuthology.orchestra.run.smithi177.stdout:4842 read 0x1188684 thru 0x118e0fd (0x5a7a bytes) 2024-05-08T03:20:43.382 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.382 INFO:teuthology.orchestra.run.smithi177.stdout:4845 read 0xb9b9f1 thru 0xba583b (0x9e4b bytes) 2024-05-08T03:20:43.383 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.383 INFO:teuthology.orchestra.run.smithi177.stdout:4848 read 0xedd718 thru 0xee8812 (0xb0fb bytes) 2024-05-08T03:20:43.383 INFO:teuthology.orchestra.run.smithi177.stdout:4849 read 0xf09473 thru 0xf14fac (0xbb3a bytes) 2024-05-08T03:20:43.386 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:43.398 INFO:teuthology.orchestra.run.smithi177.stdout:4850 write 0xad07f88 thru 0xad102a5 (0x831e bytes) 2024-05-08T03:20:43.404 INFO:teuthology.orchestra.run.smithi177.stdout:4851 trunc from 0xad102a6 to 0xc589a51 2024-05-08T03:20:43.409 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.409 INFO:teuthology.orchestra.run.smithi177.stdout:4854 read 0x550c3ca thru 0x551b1f6 (0xee2d bytes) 2024-05-08T03:20:43.409 INFO:teuthology.orchestra.run.smithi177.stdout:4855 read 0x3f45cf4 thru 0x3f471cd (0x14da bytes) 2024-05-08T03:20:43.409 INFO:teuthology.orchestra.run.smithi177.stdout:4856 writesame 0x828195c thru 0x828421e data_size 0x1(0x28c3 bytes) 2024-05-08T03:20:43.415 INFO:teuthology.orchestra.run.smithi177.stdout:4858 write 0x6751c22 thru 0x675982f (0x7c0e bytes) 2024-05-08T03:20:43.419 INFO:teuthology.orchestra.run.smithi177.stdout:4860 read 0xa89d41 thru 0xa97582 (0xd842 bytes) 2024-05-08T03:20:43.419 INFO:teuthology.orchestra.run.smithi177.stdout:4861 writesame 0xba71cd2 thru 0xba76647 data_size 0x1(0x4976 bytes) 2024-05-08T03:20:43.439 INFO:teuthology.orchestra.run.smithi177.stdout:4863 write 0xee6256a thru 0xee6b27f (0x8d16 bytes) 2024-05-08T03:20:43.442 INFO:teuthology.orchestra.run.smithi177.stdout:4864 write 0x96eb8a2 thru 0x96ecd92 (0x14f1 bytes) 2024-05-08T03:20:43.459 INFO:teuthology.orchestra.run.smithi177.stdout:4865 trunc from 0xee6b280 to 0x2bb16b9 2024-05-08T03:20:43.470 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:43.468+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:43.480 INFO:teuthology.orchestra.run.smithi177.stdout:4866 write 0x71bb446 thru 0x71bdac5 (0x2680 bytes) 2024-05-08T03:20:43.483 INFO:teuthology.orchestra.run.smithi177.stdout:4867 read 0x3d51f4b thru 0x3d5a4c7 (0x857d bytes) 2024-05-08T03:20:43.484 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:43.484 INFO:teuthology.orchestra.run.smithi177.stdout:4869 clone 37 order 18 su 16384 sc 4 2024-05-08T03:20:43.533 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.534 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.542 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:43.559 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:43.557+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:43.660 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:43.658+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:43.686 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.686 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.695 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:43.760 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:43.759+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:43.800 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.801 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.802 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:20:43.811 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:43.854 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone36 from 0x71bdac6 (overlap 0x1c3e7dc) to 0x0 2024-05-08T03:20:43.861 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:43.860+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:43.962 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:43.960+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:43.974 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:43.974 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:43.986 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:44.063 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:44.061+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:44.091 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.091 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.100 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:44.164 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:44.163+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:44.204 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.205 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.214 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:44.264 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:44.263+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:44.319 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.320 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.328 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:44.369 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:44.367+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:44.435 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.435 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.443 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:44.487 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:44.486+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:44.548 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.548 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.556 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:44.567 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:44.566+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:44.668 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:44.667+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:44.703 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.703 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.712 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:44.768 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:44.767+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:44.824 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.824 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.834 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:44.868 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-05-08T03:20:44.871 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:44.869+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:44.971 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:44.969+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:44.989 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:44.989 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:44.999 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:45.029 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:45.072 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:45.071+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:45.126 INFO:teuthology.orchestra.run.smithi177.stdout:4870 read 0x6eeb465 thru 0x6ef2cad (0x7849 bytes) 2024-05-08T03:20:45.126 INFO:teuthology.orchestra.run.smithi177.stdout:4872 read 0xa28caf thru 0xa33031 (0xa383 bytes) 2024-05-08T03:20:45.127 INFO:teuthology.orchestra.run.smithi177.stdout:4873 punch from 0x746aa9 to 0x749c74, (0x31cb bytes) 2024-05-08T03:20:45.131 INFO:teuthology.orchestra.run.smithi177.stdout:4875 clone 38 order 20 su 16384 sc 2 2024-05-08T03:20:45.151 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:45.151 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:45.161 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:45.167 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:45.167 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:45.172 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:45.171+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:45.186 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:45.186 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:45.203 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:45.203 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:45.220 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:45.220 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:45.238 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:45.238 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:45.238 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:45.255 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:45.256 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:45.268 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:45.268 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:45.269 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:45.269 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:45.274 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:45.274 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:45.274 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:45.273+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:45.276 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:45.277 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:45.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:45.375 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:45.373+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:45.423 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:45.423 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:45.423 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:45.424 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:45.424 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:45.432 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:45.476 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:45.474+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:45.576 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:45.575+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:45.579 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.477019312000003, 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:45.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.583 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.475298842000001, 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.584 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:45.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:45.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:45.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.475265401000001, 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.588 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:45.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:45.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:45.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.473839662000003, 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.473747048, 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.473236374000003, 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.473208866, 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.472802434999998, 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.472347057, 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471818810000002, 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471346312000001, 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:45.624 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471268180000003, 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471192758000001, 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471115595000001, 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471038071999999, 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.471011160000003, 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.470989950000003, 2024-05-08T03:20:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.470970170000001, 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.470415643999999, 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 37.470396632000003, 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:45.669 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:45.670 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:45.713 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:45.712+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:45.776 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:45.776 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:45.777 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:45.778 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:45.777+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:45.785 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:45.794 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 6, 4] out_osds: [7, 0, 5] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:45.794 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:45.795 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [1, 2, 3, 6, 4] 2024-05-08T03:20:45.795 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-05-08T03:20:45.821 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:20:45.878 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:45.877+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:45.884 INFO:teuthology.orchestra.run.smithi177.stdout:flattening image image_client.0-clone37 2024-05-08T03:20:45.969 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:45.969 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:45.969 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:45.969 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:45.969 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:45.980 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:45.978+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:45.981 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:46.071 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-05-08T03:20:46.073 INFO:teuthology.orchestra.run.smithi177.stdout:4876 read 0x6db0658 thru 0x6db4fb4 (0x495d bytes) 2024-05-08T03:20:46.073 INFO:teuthology.orchestra.run.smithi177.stdout:4877 read 0xd5e4bc thru 0xd6a2e1 (0xbe26 bytes) 2024-05-08T03:20:46.080 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:46.079+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:46.083 INFO:teuthology.orchestra.run.smithi177.stdout:4880 write 0x7b57e9a thru 0x7b620f2 (0xa259 bytes) 2024-05-08T03:20:46.104 INFO:teuthology.orchestra.run.smithi177.stdout:4881 write 0xdd1394f thru 0xdd17d8e (0x4440 bytes) 2024-05-08T03:20:46.109 INFO:teuthology.orchestra.run.smithi177.stdout:4882 writesame 0x6430e80 thru 0x643135f data_size 0xd(0x4e0 bytes) 2024-05-08T03:20:46.113 INFO:teuthology.orchestra.run.smithi177.stdout:4886 writesame 0x761891d thru 0x7618f09 data_size 0x1(0x5ed bytes) 2024-05-08T03:20:46.117 INFO:teuthology.orchestra.run.smithi177.stdout:4887 read 0x5def761 thru 0x5df3c4c (0x44ec bytes) 2024-05-08T03:20:46.122 INFO:teuthology.orchestra.run.smithi177.stdout:4888 write 0xddad858 thru 0xddba3e6 (0xcb8f bytes) 2024-05-08T03:20:46.124 INFO:teuthology.orchestra.run.smithi177.stdout:4890 punch from 0x425a9f7 to 0x42616aa, (0x6cb3 bytes) 2024-05-08T03:20:46.124 INFO:teuthology.orchestra.run.smithi177.stdout:4892 write 0xbc63282 thru 0xbc64510 (0x128f bytes) 2024-05-08T03:20:46.129 INFO:teuthology.orchestra.run.smithi177.stdout:4893 read 0x9020d79 thru 0x9029a22 (0x8caa bytes) 2024-05-08T03:20:46.129 INFO:teuthology.orchestra.run.smithi177.stdout:4894 write 0x76e7aa7 thru 0x76f24ef (0xaa49 bytes) 2024-05-08T03:20:46.135 INFO:teuthology.orchestra.run.smithi177.stdout:4896 trunc from 0xddba3e7 to 0xc0097c3 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:46.143 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.144728893999996, 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.144661429999999, 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:46.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:46.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:46.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.084827097999998, 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.155 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:46.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:46.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:46.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:46.160 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 46.018319085000002, 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:46.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.991781439, 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:46.165 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:46.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.963869670999998, 2024-05-08T03:20:46.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:46.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.885289478000001, 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.174 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:46.178 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880662101999999, 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:20:46.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:20:46.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880641467, 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880619215999999, 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:20:46.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880598785, 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:20:46.190 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:46.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:20:46.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:20:46.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880578559, 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:20:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880557764999999, 2024-05-08T03:20:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.880531750999999, 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:20:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:20:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.700574831000001, 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:20:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:20:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.700555727000001, 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:20:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:20:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:20:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:20:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.699994152999999, 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.694780292000001, 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:20:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 17.694612951, 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 9.3318593540000006, 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:20:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:20:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:46.233 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:46.238 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:46.238 INFO:teuthology.orchestra.run.smithi177.stdout:4900 punch from 0x6b0a5e2 to 0x6b198d5, (0xf2f3 bytes) 2024-05-08T03:20:46.238 INFO:teuthology.orchestra.run.smithi177.stdout:4901 write 0x1547110 thru 0x15499b8 (0x28a9 bytes) 2024-05-08T03:20:46.238 INFO:teuthology.orchestra.run.smithi177.stdout:4902 read 0x786cda7 thru 0x787758a (0xa7e4 bytes) 2024-05-08T03:20:46.238 INFO:teuthology.orchestra.run.smithi177.stdout:4903 read 0x7f268c2 thru 0x7f31a1c (0xb15b bytes) 2024-05-08T03:20:46.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:46.240 INFO:teuthology.orchestra.run.smithi177.stdout:4904 write 0xc04ec62 thru 0xc05d5c8 (0xe967 bytes) 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4905 trunc from 0xc05d5c9 to 0xae773f8 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4906 trunc from 0xae773f8 to 0xb3da71d 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4907 writesame 0x8b716ac thru 0x8b78eb1 data_size 0x1b(0x7806 bytes) 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4909 read 0xa1994e1 thru 0xa19c4af (0x2fcf bytes) 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4910 write 0xe164af8 thru 0xe16902f (0x4538 bytes) 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4912 read 0x5a2716 thru 0x5ad3ff (0xacea bytes) 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4913 writesame 0x7d185b thru 0x7dc88b data_size 0x61(0xb031 bytes) 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:46.241 INFO:teuthology.orchestra.run.smithi177.stdout:4917 writesame 0x39e1d1f thru 0x39e63e7 data_size 0x1(0x46c9 bytes) 2024-05-08T03:20:46.241 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:20:46.209+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e93 definitely_dead 0 2024-05-08T03:20:46.242 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:46.242 INFO:teuthology.orchestra.run.smithi177.stdout:4921 write 0xc3755c0 thru 0xc383b08 (0xe549 bytes) 2024-05-08T03:20:46.242 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:46.242 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:46.234+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:46.245 INFO:teuthology.orchestra.run.smithi177.stdout:4923 trunc from 0xe169030 to 0x62aecb 2024-05-08T03:20:46.269 INFO:teuthology.orchestra.run.smithi177.stdout:4924 read 0x20daf5 thru 0x217dd7 (0xa2e3 bytes) 2024-05-08T03:20:46.269 INFO:teuthology.orchestra.run.smithi177.stdout:4925 punch from 0x58bd49 to 0x599200, (0xd4b7 bytes) 2024-05-08T03:20:46.290 INFO:teuthology.orchestra.run.smithi177.stdout:4926 write 0xb9403bf thru 0xb941ca3 (0x18e5 bytes) 2024-05-08T03:20:46.294 INFO:teuthology.orchestra.run.smithi177.stdout:4927 writesame 0x143bbfe thru 0x1449104 data_size 0xd(0xd507 bytes) 2024-05-08T03:20:46.301 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:46.301 INFO:teuthology.orchestra.run.smithi177.stdout:4929 read 0xee3828 thru 0xee65d2 (0x2dab bytes) 2024-05-08T03:20:46.305 INFO:teuthology.orchestra.run.smithi177.stdout:4931 trunc from 0xb941ca4 to 0x8074ce0 2024-05-08T03:20:46.315 INFO:teuthology.orchestra.run.smithi177.stdout:4932 read 0x7dbb484 thru 0x7dc8606 (0xd183 bytes) 2024-05-08T03:20:46.327 INFO:teuthology.orchestra.run.smithi177.stdout:4934 write 0xc340635 thru 0xc34fae4 (0xf4b0 bytes) 2024-05-08T03:20:46.336 INFO:teuthology.orchestra.run.smithi177.stdout:4935 write 0x543cac5 thru 0x544be3d (0xf379 bytes) 2024-05-08T03:20:46.336 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:46.334+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:46.345 INFO:teuthology.orchestra.run.smithi177.stdout:4936 write 0x6476465 thru 0x6479824 (0x33c0 bytes) 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:104453 3.19 e93/91 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:head v 93'254, mlcod=93'251)", 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:46.350200+0000", 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 0.0018851440000000001, 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0019004880000000001, 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.353 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350200+0000", 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350200+0000", 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.354 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350201+0000", 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2500000000000005e-07 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350232+0000", 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1690000000000003e-05 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350245+0000", 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2775000000000001e-05 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350252+0000", 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6220000000000003e-06 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350295+0000", 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3649999999999997e-05 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:46.350309+0000", 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3101e-05 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.356 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.357 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.357 INFO:teuthology.orchestra.run.smithi188.stdout: ], 2024-05-08T03:20:46.357 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 1 2024-05-08T03:20:46.357 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:46.357 INFO:teuthology.orchestra.run.smithi177.stdout:4938 writesame 0x3f757c4 thru 0x3f7896f data_size 0x2c(0x31ac bytes) 2024-05-08T03:20:46.362 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:46.373 INFO:teuthology.orchestra.run.smithi177.stdout:4939 write 0xe0d1581 thru 0xe0d2fb8 (0x1a38 bytes) 2024-05-08T03:20:46.377 INFO:teuthology.orchestra.run.smithi177.stdout:4940 read 0x983f77a thru 0x983fb31 (0x3b8 bytes) 2024-05-08T03:20:46.377 INFO:teuthology.orchestra.run.smithi177.stdout:4941 read 0x85176e6 thru 0x851a98d (0x32a8 bytes) 2024-05-08T03:20:46.377 INFO:teuthology.orchestra.run.smithi177.stdout:4942 punch from 0x19b8ca to 0x1a9ac2, (0xe1f8 bytes) 2024-05-08T03:20:46.377 INFO:teuthology.orchestra.run.smithi177.stdout:4944 read 0x848de83 thru 0x8499cfd (0xbe7b bytes) 2024-05-08T03:20:46.377 INFO:teuthology.orchestra.run.smithi177.stdout:4945 clone 39 order 21 su 16384 sc 9 2024-05-08T03:20:46.436 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:46.435+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:46.467 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:46.467 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:46.467 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:46.468 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:46.468 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:46.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:46.538 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:46.537+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101584 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.841941+0000", 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7782938489999998, 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035511362999999997, 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.623 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101584 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841941+0000", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841941+0000", 2024-05-08T03:20:46.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841941+0000", 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841942+0000", 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5099999999999998e-07 2024-05-08T03:20:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841945+0000", 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.768e-06 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841952+0000", 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9310000000000002e-06 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877348+0000", 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035396381999999997 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877391+0000", 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2942999999999999e-05 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877452+0000", 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1179000000000005e-05 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101585 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.841954+0000", 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7782801209999999, 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035650765000000001, 2024-05-08T03:20:46.628 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101585 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841954+0000", 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841954+0000", 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841955+0000", 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4900000000000002e-07 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841956+0000", 2024-05-08T03:20:46.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.063e-06 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841959+0000", 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.869e-06 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841961+0000", 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.339e-06 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877480+0000", 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035518999000000002 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877521+0000", 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1378000000000003e-05 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877605+0000", 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3868000000000004e-05 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101586 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.841963+0000", 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7782710530000001, 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035848587000000001, 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.633 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101586 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841963+0000", 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841963+0000", 2024-05-08T03:20:46.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841964+0000", 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841965+0000", 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9400000000000004e-07 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841969+0000", 2024-05-08T03:20:46.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7859999999999996e-06 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841972+0000", 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877639+0000", 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035666949000000003 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877718+0000", 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8665999999999995e-05 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877812+0000", 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4462999999999993e-05 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101587 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.841975+0000", 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7782597630000003, 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035953129, 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.638 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101587 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841975+0000", 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841975+0000", 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841975+0000", 2024-05-08T03:20:46.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841976+0000", 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5600000000000004e-07 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841979+0000", 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7760000000000002e-06 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841981+0000", 2024-05-08T03:20:46.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3429999999999998e-06 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877838+0000", 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03585725 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877881+0000", 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2262999999999998e-05 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877928+0000", 2024-05-08T03:20:46.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7333999999999998e-05 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101589 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.841984+0000", 2024-05-08T03:20:46.642 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.778250914, 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036071350000000002, 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101589 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841984+0000", 2024-05-08T03:20:46.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841984+0000", 2024-05-08T03:20:46.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841984+0000", 2024-05-08T03:20:46.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3300000000000001e-07 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841985+0000", 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2699999999999998e-07 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841987+0000", 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7089999999999998e-06 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.841990+0000", 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.877958+0000", 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035967983000000002 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878002+0000", 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4342999999999999e-05 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878055+0000", 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2834999999999998e-05 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101703 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.842736+0000", 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7774988719999998, 2024-05-08T03:20:46.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035813279000000003, 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101703 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842736+0000", 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842736+0000", 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842736+0000", 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842738+0000", 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9580000000000001e-06 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842745+0000", 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5459999999999997e-06 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842749+0000", 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6500000000000004e-06 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878080+0000", 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035330911999999999 2024-05-08T03:20:46.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878118+0000", 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7654000000000002e-05 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878549+0000", 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00043116399999999999 2024-05-08T03:20:46.653 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101704 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 196608~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.842755+0000", 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7774793620000002, 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036388252000000003, 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.654 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101704 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842755+0000", 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842755+0000", 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842755+0000", 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2599999999999998e-07 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842757+0000", 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8980000000000001e-06 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842763+0000", 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4020000000000001e-06 2024-05-08T03:20:46.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842774+0000", 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1742999999999999e-05 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878583+0000", 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035808850000000003 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878662+0000", 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9139000000000002e-05 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879143+0000", 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048089399999999999 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.658 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101705 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 393216~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.842781+0000", 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7774538340000001, 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036865963000000002, 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101705 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842781+0000", 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842781+0000", 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842781+0000", 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6100000000000002e-07 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842783+0000", 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.069e-06 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842790+0000", 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0400000000000004e-06 2024-05-08T03:20:46.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.842794+0000", 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3069999999999996e-06 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879174+0000", 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036379881000000003 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879224+0000", 2024-05-08T03:20:46.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0104e-05 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879647+0000", 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00042220099999999998 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.663 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101775 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 4587520~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.843483+0000", 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7767511799999998, 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036012631000000003, 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101775 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.664 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843483+0000", 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843483+0000", 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843484+0000", 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.22e-07 2024-05-08T03:20:46.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843484+0000", 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8999999999999995e-07 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843487+0000", 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6979999999999998e-06 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843489+0000", 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:20:46.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.878923+0000", 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035433768999999997 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879011+0000", 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7399999999999997e-05 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879496+0000", 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048530999999999997 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.667 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101776 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 4784128~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.843492+0000", 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7767424150000002, 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036426650999999997, 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101776 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.668 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843492+0000", 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843492+0000", 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843492+0000", 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843493+0000", 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9700000000000005e-07 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843496+0000", 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6740000000000001e-06 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843498+0000", 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.356e-06 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879533+0000", 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036034507 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879608+0000", 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5043000000000006e-05 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879919+0000", 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031096500000000002 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101777 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 7405568~53248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.843501+0000", 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7767336670000002, 2024-05-08T03:20:46.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036569833000000003, 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101777 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843501+0000", 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843501+0000", 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843501+0000", 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843502+0000", 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9299999999999996e-07 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843507+0000", 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4490000000000001e-06 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843509+0000", 2024-05-08T03:20:46.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.649e-06 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879949+0000", 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036439438999999997 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.879997+0000", 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8529000000000002e-05 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880071+0000", 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3479000000000006e-05 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101778 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 4980736~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.843512+0000", 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7767223659999996, 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.037035339, 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.677 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101778 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843512+0000", 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.843512+0000", 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844444+0000", 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00093174100000000004 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844450+0000", 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7849999999999997e-06 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844459+0000", 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4339999999999998e-06 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844474+0000", 2024-05-08T03:20:46.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.4752e-05 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880105+0000", 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035631023999999997 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880180+0000", 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5403999999999998e-05 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880547+0000", 2024-05-08T03:20:46.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00036719900000000002 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101779 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 7602176~45056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.844490+0000", 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7757445279999997, 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036278188000000003, 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.682 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101779 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844490+0000", 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844490+0000", 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844491+0000", 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4700000000000001e-07 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844493+0000", 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4940000000000002e-06 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844500+0000", 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5490000000000003e-06 2024-05-08T03:20:46.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844506+0000", 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2020000000000001e-06 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880586+0000", 2024-05-08T03:20:46.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036080663999999998 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880663+0000", 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6967999999999999e-05 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880768+0000", 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000104764 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101780 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 5177344~53248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.844513+0000", 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7757212119999997, 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036792626000000002, 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101780 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.687 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844513+0000", 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844513+0000", 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844514+0000", 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.01e-07 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844516+0000", 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0930000000000001e-06 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844522+0000", 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0120000000000002e-06 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844526+0000", 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6920000000000001e-06 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880806+0000", 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036279196 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880889+0000", 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3256000000000004e-05 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881306+0000", 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000416976 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101781 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 7798784~53248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.844533+0000", 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7757018919999998, 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036909333000000003, 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.691 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101781 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844533+0000", 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844533+0000", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844533+0000", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6800000000000001e-07 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844534+0000", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2550000000000001e-06 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844537+0000", 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0850000000000001e-06 2024-05-08T03:20:46.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844540+0000", 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5969999999999999e-06 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881324+0000", 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036783859000000002 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881369+0000", 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4951999999999999e-05 2024-05-08T03:20:46.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881442+0000", 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3217000000000001e-05 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101783 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 7995392~53248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.844544+0000", 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7756907750000002, 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.037138537999999999, 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.695 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101783 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844544+0000", 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844544+0000", 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844544+0000", 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.22e-07 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844550+0000", 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6169999999999998e-06 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844553+0000", 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1310000000000002e-06 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844556+0000", 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.886e-06 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881493+0000", 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.036937801999999999 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881574+0000", 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0408000000000007e-05 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881682+0000", 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000108472 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.699 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101785 3.3 3:c0db531b:::rbd_data.1182587c3504.0000000000000000:24 [sparse-read 8192000~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.844559+0000", 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7756754030000002, 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.037290073, 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101785 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.700 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844559+0000", 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844559+0000", 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844559+0000", 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3799999999999998e-07 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844565+0000", 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2460000000000003e-06 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844568+0000", 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.461e-06 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.844571+0000", 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.661e-06 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881722+0000", 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.037151441 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881774+0000", 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1322000000000001e-05 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881849+0000", 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5604000000000003e-05 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101789 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 4194304~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.845125+0000", 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7751096359999998, 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035565334999999997, 2024-05-08T03:20:46.704 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101789 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845125+0000", 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845125+0000", 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845618+0000", 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049293500000000001 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845622+0000", 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9779999999999999e-06 2024-05-08T03:20:46.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845634+0000", 2024-05-08T03:20:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1956999999999999e-05 2024-05-08T03:20:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845652+0000", 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7969e-05 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880138+0000", 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034486197000000003 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880207+0000", 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8824000000000004e-05 2024-05-08T03:20:46.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880690+0000", 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048347499999999997 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101791 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 4390912~53248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.845701+0000", 2024-05-08T03:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7745337870000002, 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035596511999999997, 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101791 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845701+0000", 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845701+0000", 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845701+0000", 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1399999999999997e-07 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845704+0000", 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2369999999999999e-06 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845710+0000", 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5549999999999999e-06 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845715+0000", 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3129999999999998e-06 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880818+0000", 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035102901999999998 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.880863+0000", 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5006999999999998e-05 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881297+0000", 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00043388399999999999 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:101792 3.e 3:74db16ee:::rbd_data.1182587c3504.0000000000000001:24 [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:40.845741+0000", 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.7744933820000002, 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.035779396999999998, 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:46.714 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 101792 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845741+0000", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.845741+0000", 2024-05-08T03:20:46.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.847402+0000", 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.001661116 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.847406+0000", 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.839e-06 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.847415+0000", 2024-05-08T03:20:46.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2820000000000002e-06 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.847420+0000", 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9019999999999998e-06 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881338+0000", 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033917557000000001 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881415+0000", 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6824999999999997e-05 2024-05-08T03:20:46.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:40.881521+0000", 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000105876 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:46.719 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:46.719 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:46.720 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:46.721 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:46.718+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:46.813 INFO:teuthology.orchestra.run.smithi177.stderr:marked out osd.3. 2024-05-08T03:20:46.820 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:20:46.824 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:46.822+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:46.830 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:46.830 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:46.830 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:46.830 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:46.839 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:46.922 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:46.920+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:46.945 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:46.945 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:46.945 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:46.945 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:46.945 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:46.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:47.022 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:47.021+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:47.061 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:47.061 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:47.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 70.850871103000003, 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:20:47.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:20:47.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:20:47.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:20:47.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:20:47.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931190586, 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:20:47.067 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931166515000001, 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931140791999994, 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:20:47.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:20:47.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931109074999995, 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:20:47.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931083971000007, 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.087 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:20:47.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:47.092 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931059165999997, 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:47.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:20:47.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.931030621999994, 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.097 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:47.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.9310081, 2024-05-08T03:20:47.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.930977760000005, 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.107 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:20:47.111 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.930956578999997, 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:20:47.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.116 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 56.118314642999998, 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:20:47.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:47.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:20:47.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.929639922, 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.124 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:20:47.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.130 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.907436578999999, 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:20:47.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.905740461000001, 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.136 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:20:47.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.886035673000002, 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:20:47.141 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:20:47.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:20:47.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:20:47.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:20:47.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.148 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.886011713000002, 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.149 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:20:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:47.155 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.845420167999997, 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:20:47.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 46.832646271000002, 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.160 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:20:47.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.164 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 27.571907336999999, 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:20:47.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:20:47.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:47.169 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:47.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:47.172 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:47.170+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:47.272 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:47.270+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:47.276 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:47.276 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:47.276 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:47.276 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:47.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:47.285 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:47.372 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:47.371+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:47.390 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:47.390 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:47.390 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:47.390 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:47.390 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:47.399 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:47.403 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:47.404 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:47.421 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:47.421 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:47.437 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:47.437 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:47.454 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:47.454 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:47.471 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:47.472 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:47.472 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:47.483 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:47.482+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:47.489 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:47.489 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:47.506 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:47.506 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:47.509 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:47.510 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:47.546 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:47.546 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:47.554 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:47.598 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:47.597+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:47.659 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:47.659 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:47.668 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:47.710 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:47.708+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:47.776 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:47.775+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:47.813 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:47.814 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:47.827 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:47.877 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:47.876+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:47.932 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:47.932 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:47.941 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:47.978 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:47.977+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:48.046 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.046 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.055 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:48.098 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:48.097+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:48.160 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.160 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.169 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:48.179 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:48.178+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:48.280 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:48.279+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:48.313 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.313 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.322 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:48.380 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:48.379+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:48.428 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.428 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.437 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:48.481 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:48.479+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:48.541 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.542 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.551 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:48.583 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:48.582+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:48.683 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:48.681+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:48.697 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.697 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:48.785 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:48.784+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:48.841 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:20:48.869 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:48.870 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:48.880 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:48.916 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone38 from 0xe0d2fb9 (overlap 0x62aecb) to 0x4ef23c 2024-05-08T03:20:48.916 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:48.914+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:48.986 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:48.985+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:48.994 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-05-08T03:20:49.056 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:49.056 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:49.067 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:49.109 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:49.107+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:49.149 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:49.149 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:49.149 INFO:teuthology.orchestra.run.smithi177.stdout:4948 write 0x2d567ca thru 0x2d5d27b (0x6ab2 bytes) 2024-05-08T03:20:49.163 INFO:teuthology.orchestra.run.smithi177.stdout:4949 read 0x14418f0 thru 0x144c386 (0xaa97 bytes) 2024-05-08T03:20:49.164 INFO:teuthology.orchestra.run.smithi177.stdout:4951 read 0xd8eb940 thru 0xd8ecb0c (0x11cd bytes) 2024-05-08T03:20:49.164 INFO:teuthology.orchestra.run.smithi177.stdout:4952 writesame 0x1a1233b thru 0x1a1b9c1 data_size 0x23(0x9687 bytes) 2024-05-08T03:20:49.182 INFO:teuthology.orchestra.run.smithi177.stdout:4953 clone 40 order 20 su 4096 sc 10 2024-05-08T03:20:49.188 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:49.186+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:49.249 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:49.249 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:49.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:49.301 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:49.300+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:49.379 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:49.379 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:49.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:49.390 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:49.389+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:49.491 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:49.490+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:49.506 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:49.506 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:49.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:49.515 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:49.592 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:49.591+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:49.633 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:49.633 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:49.643 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:49.659 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:49.659 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:49.680 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:49.680 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:49.693 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:49.692+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:49.701 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:49.701 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:49.722 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:49.722 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:49.747 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:49.748 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:49.748 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:49.753 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:49.753 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:49.762 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:49.768 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:49.768 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:49.770 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:20:49.796 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:49.796 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:49.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:49.801 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:49.806 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:49.805+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:49.866 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone39 from 0xe0d2fb9 (overlap 0xe0d2fb9) to 0x43727ea 2024-05-08T03:20:49.875 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:49.875 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:49.876 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:49.876 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:49.876 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:49.884 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:49.894 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:49.893+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:49.995 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:49.994+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:50.037 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:50.037 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:50.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:50.037 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:50.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.038 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 73.825588357000001, 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:20:50.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:20:50.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:20:50.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905907839999998, 2024-05-08T03:20:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:20:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:20:50.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.048 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905883768999999, 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.052 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905858046000006, 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:20:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:20:50.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905826329000007, 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.057 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:50.061 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905801225000005, 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:20:50.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:20:50.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905776419999995, 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:20:50.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:20:50.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:20:50.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:20:50.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905747876000007, 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:20:50.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905725353999998, 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.074 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905695014000003, 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:50.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 72.905673832999994, 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:20:50.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:20:50.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:20:50.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.093031897000003, 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:20:50.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:20:50.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:50.093 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.904357175999998, 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:50.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.882153832999997, 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.099 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.880457714999999, 2024-05-08T03:20:50.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:20:50.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:20:50.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.860752927, 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:20:50.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:20:50.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:20:50.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:20:50.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.860728967, 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:20:50.113 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.820137422000002, 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:20:50.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:20:50.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 49.807363525, 2024-05-08T03:20:50.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:20:50.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:20:50.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 30.546624591, 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:20:50.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:20:50.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:20:50.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.135 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.135 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.135 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:50.136 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:50.137 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:50.135+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:50.237 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:50.236+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:50.286 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:50.286 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:50.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:50.338 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:50.336+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:50.439 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:50.437+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:50.446 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:50.446 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:50.446 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:50.446 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:50.447 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:50.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:50.539 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:50.538+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.693575838000001, 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:20:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:20:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351634645999994, 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:20:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351619247000002, 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:20:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:20:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351587605000006, 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:20:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351574272999997, 2024-05-08T03:20:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351564479000004, 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:20:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351555066000003, 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:20:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:20:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351545544999993, 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351536203999999, 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351526989000007, 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:20:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:20:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351517627000007, 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351508194999994, 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:20:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:20:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:20:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351498778999996, 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351489760999996, 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351480718000005, 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:20:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:20:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351468424999993, 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:20:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:20:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351459157999997, 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:20:50.640 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:50.644 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351445299999995, 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:20:50.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:20:50.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.648 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.351436007000004, 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.652 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.358421874000001, 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:20:50.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:20:50.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:20:50.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:20:50.657 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.657 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.657 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.657 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:50.657 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:50.657 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:50.658 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:50.659 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:50.656+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:50.759 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:50.758+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:50.767 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:50.767 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:50.767 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:50.767 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:50.779 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:50.860 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:50.859+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:50.892 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:50.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:50.892 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:50.892 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:50.893 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:50.902 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:50.961 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:50.959+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:51.024 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.822199269999999, 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.025 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:51.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:51.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.822189820000006, 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.031 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:51.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.822177084000003, 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:51.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:51.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:51.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:51.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.041 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.822166507999995, 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:51.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.822157121000004, 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:51.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:51.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:51.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:51.050 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.822147940999997, 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:51.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:51.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:51.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591322906000002, 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:51.058 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591290669000003, 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.059 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:51.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:51.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591277339000001, 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.064 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:51.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:51.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.067 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591262365000006, 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:51.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:51.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591251056000004, 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:51.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:51.075 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591233387000003, 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.076 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:51.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:51.079 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591218189000003, 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.080 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.591204094999995, 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.084 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 74.816250122, 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:51.088 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:51.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.893356319999995, 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:51.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:51.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.893342832000002, 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.098 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.893313851000002, 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.102 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:51.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:51.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:51.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.893287244999996, 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:51.106 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:51.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:51.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.460538983999996, 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:51.110 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.116 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:51.118 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:51.118 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:51.116+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:51.218 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:51.217+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105504 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801737+0000", 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46266759000000002, 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46268554000000001, 2024-05-08T03:20:51.268 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105504 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801737+0000", 2024-05-08T03:20:51.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801737+0000", 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801738+0000", 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801739+0000", 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5039999999999999e-06 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801742+0000", 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2329999999999998e-06 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801745+0000", 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7259999999999998e-06 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801841+0000", 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5622e-05 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801845+0000", 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7740000000000004e-06 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105505 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801748+0000", 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.462656452, 2024-05-08T03:20:51.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46271312399999998, 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105505 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801748+0000", 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801748+0000", 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801749+0000", 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0100000000000001e-07 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801750+0000", 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.254e-06 2024-05-08T03:20:51.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801753+0000", 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8789999999999999e-06 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801756+0000", 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7709999999999998e-06 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801869+0000", 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000113041 2024-05-08T03:20:51.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801876+0000", 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3930000000000002e-06 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105454 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795588+0000", 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46881689199999999, 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46890644599999998, 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105454 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795588+0000", 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795588+0000", 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0359999999999999e-06 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795594+0000", 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.147e-06 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795605+0000", 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1277000000000001e-05 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795613+0000", 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1790000000000002e-06 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795676+0000", 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3252999999999999e-05 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795687+0000", 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0667e-05 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105455 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795638+0000", 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46876643899999998, 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46888743300000002, 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105455 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795638+0000", 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795638+0000", 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795639+0000", 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8699999999999995e-07 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795642+0000", 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.225e-06 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795651+0000", 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4330000000000004e-06 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795656+0000", 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4639999999999997e-06 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795721+0000", 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4475999999999997e-05 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795729+0000", 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.524e-06 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105456 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795666+0000", 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46873900299999999, 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46889150899999998, 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105456 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.284 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795666+0000", 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795666+0000", 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795666+0000", 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.99e-07 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795669+0000", 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7329999999999999e-06 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795676+0000", 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3489999999999997e-06 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795681+0000", 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.49e-06 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795779+0000", 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8275000000000004e-05 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795789+0000", 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3619999999999995e-06 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105457 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795689+0000", 2024-05-08T03:20:51.287 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46871600499999999, 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.468899605, 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105457 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795689+0000", 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795689+0000", 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795689+0000", 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7e-07 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795692+0000", 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6299999999999998e-06 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795698+0000", 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0430000000000004e-06 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795703+0000", 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1189999999999996e-06 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795841+0000", 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000137634 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795850+0000", 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3060000000000003e-06 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105458 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795709+0000", 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.468695533, 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46891123699999998, 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.291 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105458 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795709+0000", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795709+0000", 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795710+0000", 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6199999999999999e-07 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795711+0000", 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.787e-06 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795717+0000", 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7799999999999997e-06 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795722+0000", 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795888+0000", 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016606699999999999 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795897+0000", 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0040000000000005e-06 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.294 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105459 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795729+0000", 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46867572499999999, 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46892371799999999, 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105459 2024-05-08T03:20:51.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795729+0000", 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795729+0000", 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795730+0000", 2024-05-08T03:20:51.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795732+0000", 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6570000000000001e-06 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795738+0000", 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1759999999999998e-06 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795744+0000", 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3709999999999999e-06 2024-05-08T03:20:51.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795933+0000", 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018975099999999999 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795940+0000", 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4439999999999996e-06 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.298 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105460 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795765+0000", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46863961700000001, 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46891883899999998, 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105460 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795765+0000", 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795765+0000", 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795766+0000", 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5999999999999999e-07 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795768+0000", 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5000000000000002e-06 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795776+0000", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9039999999999992e-06 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795782+0000", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.541e-06 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795984+0000", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020236200000000001 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795993+0000", 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1880000000000008e-06 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105461 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.795789+0000", 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46861571499999999, 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46892615300000001, 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.302 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105461 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795789+0000", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795789+0000", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795790+0000", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5400000000000002e-07 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795792+0000", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795797+0000", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5569999999999997e-06 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.795803+0000", 2024-05-08T03:20:51.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4559999999999997e-06 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796015+0000", 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021209399999999999 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796022+0000", 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.305 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105469 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796176+0000", 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46822878499999998, 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.468570192, 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105469 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796176+0000", 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796176+0000", 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796177+0000", 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.09e-07 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796181+0000", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4680000000000003e-06 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796194+0000", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2629999999999999e-05 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796201+0000", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8220000000000001e-06 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:20:51.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6218999999999999e-05 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796255+0000", 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.763e-06 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46817894300000001, 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46855154500000001, 2024-05-08T03:20:51.309 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46815464899999998, 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46855814800000001, 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.316 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46808153200000002, 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.468516194, 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.317 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:20:51.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:20:51.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.320 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46805329899999998, 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46852175800000001, 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:20:51.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46802255199999998, 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.468522153, 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:20:51.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:20:51.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105422 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792524+0000", 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47188121599999999, 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47241191799999999, 2024-05-08T03:20:51.328 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105422 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792524+0000", 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792524+0000", 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792526+0000", 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1840000000000002e-06 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792532+0000", 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1240000000000002e-06 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5415000000000001e-05 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792555+0000", 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2910000000000002e-06 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792635+0000", 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9918000000000003e-05 2024-05-08T03:20:51.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792660+0000", 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5301000000000001e-05 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46798519300000002, 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46854696400000001, 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.332 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:20:51.335 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105423 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792575+0000", 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47182939899999998, 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47242194900000001, 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.336 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105423 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792575+0000", 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792575+0000", 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792576+0000", 2024-05-08T03:20:51.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8999999999999997e-07 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792579+0000", 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8789999999999999e-06 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792588+0000", 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8359999999999998e-06 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792593+0000", 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792737+0000", 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014322799999999999 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792745+0000", 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5969999999999992e-06 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.339 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105424 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792618+0000", 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47178690099999998, 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47241027600000002, 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105424 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.340 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792618+0000", 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792618+0000", 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792618+0000", 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0999999999999999e-07 2024-05-08T03:20:51.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792621+0000", 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6860000000000002e-06 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792627+0000", 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1020000000000002e-06 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792632+0000", 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9629999999999997e-06 2024-05-08T03:20:51.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792775+0000", 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000142915 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792782+0000", 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5300000000000002e-06 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:20:51.343 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46638346200000003, 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46703804300000001, 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:20:51.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:20:51.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105425 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792666+0000", 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47173836099999999, 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47243003, 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105425 2024-05-08T03:20:51.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792666+0000", 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792666+0000", 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792667+0000", 2024-05-08T03:20:51.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0500000000000002e-07 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792672+0000", 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5510000000000001e-06 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792684+0000", 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.205e-05 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792692+0000", 2024-05-08T03:20:51.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.14e-06 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792833+0000", 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014111500000000001 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792844+0000", 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0685e-05 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46632944100000001, 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46707631999999999, 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:20:51.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47170053699999998, 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.472480491, 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46629673500000002, 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46710905200000002, 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:20:51.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:20:51.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:20:51.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.361 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47167451199999999, 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47252669000000003, 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:20:51.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46627234299999998, 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46715844000000001, 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.365 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47163249699999998, 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.472550992, 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.369 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46625292200000001, 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46724637400000002, 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.373 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:51.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:20:51.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.376 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46624164299999998, 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46726625199999999, 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:20:51.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:51.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:20:51.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105437 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.380 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.470863158, 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47192003100000002, 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105437 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8299999999999995e-07 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793547+0000", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6890000000000003e-06 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793560+0000", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2747e-05 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793569+0000", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.524e-06 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793627+0000", 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8294000000000001e-05 2024-05-08T03:20:51.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793636+0000", 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1360000000000003e-06 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46622194900000002, 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46730943800000002, 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.384 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:20:51.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105438 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793599+0000", 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.470806314, 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47192589499999998, 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.388 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105438 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793599+0000", 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793599+0000", 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793599+0000", 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0299999999999999e-07 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793603+0000", 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4529999999999998e-06 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793611+0000", 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5720000000000003e-06 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793618+0000", 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8000000000000001e-06 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793676+0000", 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8075000000000002e-05 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793685+0000", 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2480000000000002e-06 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105439 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793629+0000", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47077614899999998, 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47192741300000002, 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105439 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793629+0000", 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793629+0000", 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793629+0000", 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793632+0000", 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7360000000000001e-06 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793639+0000", 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4499999999999998e-06 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793645+0000", 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7899999999999996e-06 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793727+0000", 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2317999999999993e-05 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793738+0000", 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0659000000000001e-05 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105499 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801592+0000", 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46281258199999997, 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.463994552, 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.395 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105499 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801592+0000", 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801592+0000", 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801593+0000", 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2900000000000003e-07 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801596+0000", 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4999999999999999e-06 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801609+0000", 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2368000000000001e-05 2024-05-08T03:20:51.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801616+0000", 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6880000000000002e-06 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801668+0000", 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2083000000000003e-05 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801677+0000", 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.4769999999999992e-06 2024-05-08T03:20:51.398 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105440 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793656+0000", 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47074913600000001, 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47196297599999998, 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.399 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105440 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793656+0000", 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793656+0000", 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793656+0000", 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0400000000000002e-07 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793658+0000", 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1749999999999999e-06 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793683+0000", 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4287999999999999e-05 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793690+0000", 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5279999999999998e-06 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793788+0000", 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.8109000000000005e-05 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793796+0000", 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.977e-06 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105500 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801630+0000", 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46277505699999999, 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.464019338, 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105500 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801630+0000", 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801630+0000", 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801630+0000", 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5299999999999999e-07 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801633+0000", 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5050000000000002e-06 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801639+0000", 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1149999999999999e-06 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801644+0000", 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8180000000000003e-06 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801705+0000", 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1644999999999997e-05 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801713+0000", 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5449999999999998e-06 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:20:51.406 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47064982100000002, 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47192590299999998, 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105501 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801663+0000", 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46274152000000002, 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.464049402, 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.410 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105501 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801663+0000", 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801663+0000", 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801664+0000", 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801666+0000", 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3719999999999999e-06 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801672+0000", 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7520000000000002e-06 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801682+0000", 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6490000000000003e-06 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801739+0000", 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7807999999999999e-05 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801747+0000", 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5120000000000002e-06 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.413 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.470613905, 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.47195252599999998, 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:20:51.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:51.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:20:51.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105502 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801703+0000", 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.46270155600000001, 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46407189100000001, 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105502 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801703+0000", 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801703+0000", 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801704+0000", 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1800000000000002e-07 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801706+0000", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0770000000000002e-06 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801710+0000", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4220000000000002e-06 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801714+0000", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1200000000000004e-06 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801772+0000", 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7732000000000002e-05 2024-05-08T03:20:51.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801776+0000", 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5050000000000001e-06 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.47058886300000002, 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.471990417, 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.421 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:51.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:20:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:20:51.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105503 3.19 3.bf694759 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.801721+0000", 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.462684031, 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.46411746500000001, 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "waiting for peered", 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105503 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801721+0000", 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801721+0000", 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801721+0000", 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3099999999999999e-07 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801722+0000", 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.44e-06 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801726+0000", 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2140000000000001e-06 2024-05-08T03:20:51.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801729+0000", 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9079999999999999e-06 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801808+0000", 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9108000000000002e-05 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.801816+0000", 2024-05-08T03:20:51.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3790000000000006e-06 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 43 2024-05-08T03:20:51.429 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:51.430 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:51.431 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:51.428+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:51.531 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:51.530+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:51.576 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:51.577 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:51.577 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:51.577 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:51.577 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:51.586 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:51.669 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:51.668+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:51.733 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:51.731+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:17115 3.e 3:707e5f06:::rbd_object_map.118231c091d8:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:51.734 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.946728569000001, 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.40677507800000001, 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 17115 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:51.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785294+0000", 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785296+0000", 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2060000000000001e-06 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785304+0000", 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9279999999999993e-06 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785320+0000", 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6529999999999999e-05 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785329+0000", 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4840000000000004e-06 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785385+0000", 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6505000000000002e-05 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:45.785393+0000", 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6399999999999997e-06 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190458+0000", 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.405065546 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190617+0000", 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00015834900000000001 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 0", 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.190824+0000", 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00020745400000000001 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191958+0000", 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011337509999999999 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.739 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.191970+0000", 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.2035000000000001e-05 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192007+0000", 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6483000000000003e-05 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:46.192069+0000", 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2167000000000003e-05 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.799494420000002, 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:51.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:51.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23274 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:51.747 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.609354252999999, 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17044426500000001, 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23274 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122668+0000", 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122671+0000", 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8820000000000001e-06 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147335+0000", 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.024663898 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147364+0000", 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9213e-05 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147375+0000", 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0631999999999999e-05 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147451+0000", 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6796000000000001e-05 2024-05-08T03:20:51.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147574+0000", 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012270099999999999 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.151611+0000", 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0040368410000000002 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.273617+0000", 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.12200573100000001 2024-05-08T03:20:51.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.292984+0000", 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019367266000000001 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293052+0000", 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7988e-05 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.293112+0000", 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0316999999999999e-05 2024-05-08T03:20:51.752 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23275 3.1c 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head [truncate 19262493] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.584613707000003, 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.16473921899999999, 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.753 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23275 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147408+0000", 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147410+0000", 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8789999999999999e-06 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147416+0000", 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3120000000000001e-06 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147422+0000", 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9290000000000003e-06 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.147427+0000", 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0679999999999996e-06 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207158+0000", 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059731546000000003 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207258+0000", 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9611999999999994e-05 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.207418+0000", 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016024400000000001 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.283058+0000", 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075639838000000001 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312026+0000", 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.02896775 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312094+0000", 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8218000000000003e-05 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.312148+0000", 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3823000000000003e-05 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.758 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67863 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.494681219, 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069622329999999996, 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67863 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237341+0000", 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-07 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237344+0000", 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237349+0000", 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7339999999999996e-06 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237354+0000", 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.605e-06 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306659+0000", 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069304893000000006 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306767+0000", 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000108385 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306963+0000", 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00019608200000000001 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.762 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67864 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.494662243000001, 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069780320000000007, 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67864 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237360+0000", 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237362+0000", 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0679999999999999e-06 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237369+0000", 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4269999999999996e-06 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237373+0000", 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.296e-06 2024-05-08T03:20:51.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.306990+0000", 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069617100000000001 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307041+0000", 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0796e-05 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307140+0000", 2024-05-08T03:20:51.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9189000000000002e-05 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67865 3.1b 3:dea95625:::rbd_data.11821374e988.0000000000000002:21 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.494643637999999, 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070114813999999998, 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.767 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67865 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237379+0000", 2024-05-08T03:20:51.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.96e-07 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237381+0000", 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8670000000000001e-06 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237386+0000", 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.48e-06 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237392+0000", 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5620000000000003e-06 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307171+0000", 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.069779439999999998 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307237+0000", 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5838000000000002e-05 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307493+0000", 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00025623099999999999 2024-05-08T03:20:51.770 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67866 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.494624339000001, 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070792377000000004, 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.771 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67866 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237398+0000", 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5999999999999999e-07 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237401+0000", 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7630000000000001e-06 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237407+0000", 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7169999999999996e-06 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237412+0000", 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4729999999999999e-06 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307524+0000", 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070111455000000003 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.307595+0000", 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1178000000000002e-05 2024-05-08T03:20:51.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308190+0000", 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000595431 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67867 3.1b 3:df4ecfbf:::rbd_data.11821374e988.0000000000000003:21 [read 16384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.494603949999998, 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072514764999999995, 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.775 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67867 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237418+0000", 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237419+0000", 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7899999999999999e-07 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237421+0000", 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9860000000000001e-06 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237426+0000", 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.614e-06 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237431+0000", 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2680000000000002e-06 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308233+0000", 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070801136000000001 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.308945+0000", 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00071273699999999996 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309933+0000", 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0009876450000000001 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67868 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.494584781, 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072516667000000007, 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67868 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.237437+0000", 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239091+0000", 2024-05-08T03:20:51.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001654006 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239096+0000", 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.921e-06 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239107+0000", 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0781000000000001e-05 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239114+0000", 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.939e-06 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309950+0000", 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070835871999999994 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309954+0000", 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1479999999999999e-06 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.783 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67869 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492895197999999, 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070844969999999993, 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67869 2024-05-08T03:20:51.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239127+0000", 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239128+0000", 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239130+0000", 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.655e-06 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239138+0000", 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2690000000000002e-06 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239143+0000", 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6690000000000003e-06 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309966+0000", 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070823102999999998 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309972+0000", 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6350000000000003e-06 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.787 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67870 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492871470000001, 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070850033000000007, 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.788 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67870 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239151+0000", 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239153+0000", 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2189999999999998e-06 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239160+0000", 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7170000000000002e-06 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239166+0000", 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4140000000000002e-06 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.309994+0000", 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070828156000000003 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310001+0000", 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0310000000000001e-06 2024-05-08T03:20:51.791 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67871 3.1b 3.fdf372fb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492849548999999, 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070908609999999997, 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.792 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67871 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239173+0000", 2024-05-08T03:20:51.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239175+0000", 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1909999999999999e-06 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239182+0000", 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5640000000000002e-06 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239187+0000", 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9409999999999998e-06 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310071+0000", 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070883710000000003 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310081+0000", 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0716999999999999e-05 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.795 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67876 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492739104000002, 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070745004, 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67876 2024-05-08T03:20:51.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239283+0000", 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239286+0000", 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0329999999999998e-06 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239291+0000", 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.57e-06 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239296+0000", 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5789999999999997e-06 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310022+0000", 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070725882000000004 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310028+0000", 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5690000000000001e-06 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.799 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67877 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492720279, 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070752494999999999, 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67877 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.800 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239302+0000", 2024-05-08T03:20:51.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5499999999999999e-07 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239305+0000", 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239311+0000", 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9100000000000002e-06 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239316+0000", 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.527e-06 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310048+0000", 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070732489999999995 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310054+0000", 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2269999999999998e-06 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67878 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492700411000001, 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070795459000000005, 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67878 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.804 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239322+0000", 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8799999999999998e-07 2024-05-08T03:20:51.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239324+0000", 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239330+0000", 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8050000000000003e-06 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239335+0000", 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5029999999999999e-06 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310110+0000", 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070775530000000003 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310117+0000", 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2089999999999999e-06 2024-05-08T03:20:51.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67879 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492681899000001, 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070805241000000005, 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67879 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239340+0000", 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239341+0000", 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239343+0000", 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9400000000000001e-06 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239349+0000", 2024-05-08T03:20:51.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.533e-06 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239354+0000", 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4549999999999997e-06 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310138+0000", 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070784480999999996 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310146+0000", 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:67880 3.3 3.121d5e63 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 24.492432538999999, 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070583059000000004, 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 67880 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239590+0000", 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2200000000000003e-07 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239593+0000", 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8059999999999999e-06 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239600+0000", 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.584e-06 2024-05-08T03:20:51.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.239605+0000", 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6699999999999999e-06 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310166+0000", 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070560237999999997 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:27.310173+0000", 2024-05-08T03:20:51.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0389999999999999e-06 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:80533 3.f 3:f6536522:::rbd_data.11828c8e13f1.000000000000002f:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.797519139999999, 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083900980999999999, 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 80533 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934503+0000", 2024-05-08T03:20:51.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934504+0000", 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8299999999999996e-07 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934509+0000", 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8520000000000003e-06 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934516+0000", 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.058e-06 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.934522+0000", 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2029999999999997e-06 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.935441+0000", 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000918745 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.935580+0000", 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013952300000000001 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.018404+0000", 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.082823816999999994 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:80614 3.16 3:696d6acd:::rbd_data.11828c8e13f1.0000000000000030:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:51.820 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.791959509000002, 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.078694985999999995, 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 80614 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940063+0000", 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6699999999999999e-07 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940066+0000", 2024-05-08T03:20:51.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8329999999999998e-06 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940073+0000", 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1840000000000002e-06 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.940079+0000", 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7139999999999998e-06 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:51.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.943185+0000", 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0031060459999999999 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.943333+0000", 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014836 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:29.018758+0000", 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075424381999999998 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.824 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:51.825 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:51.825 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:51.825 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:20:51.825 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:51.827 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:51.836 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:51.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:20:51.836 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T03:20:51.836 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:20:51.870 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:51.867+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:51.934 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:51.933+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:51.996 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:51.997 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:52.015 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:52.016 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:52.035 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:52.035 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:52.035 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:52.033+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:52.040 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.040 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.050 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:52.053 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:52.053 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:52.086 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:52.087 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:52.087 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:52.105 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:52.105 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:52.122 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:52.123 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:52.126 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:52.126 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:52.160 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.160 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.160 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:52.158+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:52.169 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:52.237 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:52.236+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:52.238 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:20:52.238 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:52.238 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 99, 2024-05-08T03:20:52.238 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:20:52.238 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:20:52.238 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:20:51.772500+0000", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:20:08.032979+0000", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:20:46.806790+0000", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5804032, 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "nodeep-scrub", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 5, 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:20:52.239 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "luminous", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:52.240 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:52.241 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:52.242 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:52.243 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:52.244 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 3.5099999904632568, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 3.5099999904632568, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.56999999284744263, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:20:52.245 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:52.246 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:52.247 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:52.248 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:52.249 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:20:52.250 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 6.9000000953674316, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 6.9000000953674316, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:52.251 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:20:52.252 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:20:52.253 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "97", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 42, 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 97, 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:20:52.254 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:20:52.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:20:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 3.9500000476837158, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 3.9500000476837158, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.56999999284744263, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2.25, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2.25, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 1, 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:52.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 40, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.258 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.259 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.260 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:20:52.261 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.262 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 98, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.263 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.264 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.265 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:20:52.267 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.268 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 98, 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.269 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.270 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.272 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:52.273 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 98, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.277 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 98, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.280 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 0, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 45, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 61, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.284 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1643429991 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1643429991", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1643429991", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1643429991", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1643429991", 2024-05-08T03:20:52.288 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "exists" 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 54, 2024-05-08T03:20:52.289 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.291 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.292 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:20:52.293 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:20:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:20:52.295 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.297 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.299 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:20:52.300 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.301 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:20:52.302 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 61 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.303 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:20:52.304 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [ 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "1.5", 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2, 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: 0 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.d", 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: 6 2024-05-08T03:20:52.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.19", 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: 4 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [ 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.6", 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:52.306 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 1, 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:20:52.307 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [], 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [], 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:20:52.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:52.309 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:52.310 INFO:tasks.thrashosds.thrasher:j is {'epoch': 99, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:20:51.772500+0000', 'last_up_change': '2024-05-08T03:20:08.032979+0000', 'last_in_change': '2024-05-08T03:20:46.806790+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.509999990463257, 'score_stable': 3.509999990463257, 'optimal_score': 0.5699999928474426, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.950000047683716, 'score_stable': 3.950000047683716, 'optimal_score': 0.5699999928474426, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 45, 'down_at': 61, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1643429991}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1643429991}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1643429991}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1643429991}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1643429991}]}, 'public_addr': '172.21.15.188:6817/1643429991', 'cluster_addr': '172.21.15.188:6819/1643429991', 'heartbeat_back_addr': '172.21.15.188:6823/1643429991', 'heartbeat_front_addr': '172.21.15.188:6821/1643429991', 'state': ['exists']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 61}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.5', 'osds': [2, 0]}, {'pgid': '3.d', 'osds': [3, 6]}, {'pgid': '3.19', 'osds': [3, 4]}], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:20:52.310 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.d 2024-05-08T03:20:52.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.d 2024-05-08T03:20:52.316 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.316 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.326 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:52.338 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:52.337+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:52.439 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:52.437+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:52.507 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.507 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.520 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:52.561 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:52.559+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:52.641 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:52.640+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:52.668 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.668 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.677 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:52.743 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:52.742+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:52.789 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:20:52.790 INFO:teuthology.orchestra.run.smithi177.stderr:clear 3.d pg_upmap mapping 2024-05-08T03:20:52.836 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.836 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.844 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:52.842+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:52.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:52.939 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-05-08T03:20:52.945 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:52.944+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:52.949 INFO:teuthology.orchestra.run.smithi177.stdout:4954 write 0xc4e392e thru 0xc4ebf93 (0x8666 bytes) 2024-05-08T03:20:52.967 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:52.967 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:52.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:52.994 INFO:teuthology.orchestra.run.smithi177.stdout:4955 writesame 0x9753654 thru 0x97617f9 data_size 0x11(0xe1a6 bytes) 2024-05-08T03:20:53.016 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:53.032 INFO:teuthology.orchestra.run.smithi177.stdout:4957 trunc from 0xe0d2fb9 to 0xf51eaa 2024-05-08T03:20:53.046 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:53.045+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:53.070 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:53.070 INFO:teuthology.orchestra.run.smithi177.stdout:4959 read 0xdbd4e9 thru 0xdc660e (0x9126 bytes) 2024-05-08T03:20:53.077 INFO:teuthology.orchestra.run.smithi177.stdout:4960 write 0x285252d thru 0x285d784 (0xb258 bytes) 2024-05-08T03:20:53.093 INFO:teuthology.orchestra.run.smithi177.stdout:4961 read 0x16d936a thru 0x16e3b54 (0xa7eb bytes) 2024-05-08T03:20:53.101 INFO:teuthology.orchestra.run.smithi177.stdout:4962 writesame 0x44bebc4 thru 0x44cdae8 data_size 0x1(0xef25 bytes) 2024-05-08T03:20:53.101 INFO:teuthology.orchestra.run.smithi177.stdout:4964 read 0x281d9d thru 0x284b5a (0x2dbe bytes) 2024-05-08T03:20:53.102 INFO:teuthology.orchestra.run.smithi177.stdout:4965 read 0x1afdcc1 thru 0x1b0452b (0x686b bytes) 2024-05-08T03:20:53.111 INFO:teuthology.orchestra.run.smithi177.stdout:4967 trunc from 0x44cdae9 to 0xc2c7134 2024-05-08T03:20:53.116 INFO:teuthology.orchestra.run.smithi177.stdout:4968 write 0x6e5706d thru 0x6e65cc4 (0xec58 bytes) 2024-05-08T03:20:53.134 INFO:teuthology.orchestra.run.smithi177.stdout:4969 read 0x84598a3 thru 0x8466606 (0xcd64 bytes) 2024-05-08T03:20:53.134 INFO:teuthology.orchestra.run.smithi177.stdout:4970 punch from 0x2f12b99 to 0x2f1758d, (0x49f4 bytes) 2024-05-08T03:20:53.135 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:53.135 INFO:teuthology.orchestra.run.smithi177.stdout:4972 read 0x8e29c8f thru 0x8e37d05 (0xe077 bytes) 2024-05-08T03:20:53.135 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.135 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:53.147 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:53.146+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:53.148 INFO:teuthology.orchestra.run.smithi177.stdout:4974 trunc from 0xc2c7134 to 0xd13ce6 2024-05-08T03:20:53.172 INFO:teuthology.orchestra.run.smithi177.stdout:4976 writesame 0x705fdf thru 0x711398 data_size 0x56(0xb3ba bytes) 2024-05-08T03:20:53.247 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:53.247+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:53.252 INFO:teuthology.orchestra.run.smithi177.stdout:4977 trunc from 0xd13ce6 to 0x4de84d6 2024-05-08T03:20:53.257 INFO:teuthology.orchestra.run.smithi177.stdout:4979 punch from 0x2c27231 to 0x2c2f80c, (0x85db bytes) 2024-05-08T03:20:53.259 INFO:teuthology.orchestra.run.smithi177.stdout:4980 trunc from 0x4de84d6 to 0x6a27ad9 2024-05-08T03:20:53.264 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:53.264 INFO:teuthology.orchestra.run.smithi177.stdout:4983 writesame 0x60ebe3b thru 0x60f746c data_size 0x1(0xb632 bytes) 2024-05-08T03:20:53.274 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.274 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.282 INFO:teuthology.orchestra.run.smithi177.stdout:4985 write 0x49d8fb0 thru 0x49db966 (0x29b7 bytes) 2024-05-08T03:20:53.286 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:53.294 INFO:teuthology.orchestra.run.smithi177.stdout:4986 trunc from 0x6a27ad9 to 0x3308b76 2024-05-08T03:20:53.311 INFO:teuthology.orchestra.run.smithi177.stdout:4988 punch from 0x214ce42 to 0x2159487, (0xc645 bytes) 2024-05-08T03:20:53.327 INFO:teuthology.orchestra.run.smithi177.stdout:4990 write 0xc1d390e thru 0xc1dfe9e (0xc591 bytes) 2024-05-08T03:20:53.343 INFO:teuthology.orchestra.run.smithi177.stdout:4993 write 0x60fc3e thru 0x61c1fe (0xc5c1 bytes) 2024-05-08T03:20:53.347 INFO:teuthology.orchestra.run.smithi177.stdout:4994 read 0x2f525d0 thru 0x2f5c22c (0x9c5d bytes) 2024-05-08T03:20:53.347 INFO:teuthology.orchestra.run.smithi177.stdout:4995 clone 41 order 21 su 8192 sc 15 2024-05-08T03:20:53.348 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:53.346+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:53.434 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.435 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.444 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:53.488 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:53.487+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:53.549 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.549 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.550 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:53.548+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:53.558 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:53.651 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:53.650+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:53.703 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.703 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.713 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:53.752 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:53.751+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:53.821 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.822 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.831 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:53.857 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone40 from 0xc1dfe9f (overlap 0xd13ce6) to 0xbc5068 2024-05-08T03:20:53.875 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:53.874+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:53.945 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:53.945 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:53.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:53.956 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:53.953+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:53.957 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-05-08T03:20:54.056 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:54.054+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:54.057 INFO:teuthology.orchestra.run.smithi177.stdout:4997 flatten 2024-05-08T03:20:54.123 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:54.123 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:54.127 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:54.135 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:54.171 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:54.169+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:54.258 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:54.256+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:54.292 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:54.293 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:54.315 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:54.315 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:54.316 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:54.316 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:54.316 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:54.316 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:54.325 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:54.336 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:54.336 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:54.358 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:54.357+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:54.362 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:54.363 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:54.391 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:54.392 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:54.392 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:54.418 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:54.418 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:54.438 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:54.439 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:54.442 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:54.442 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:54.457 INFO:teuthology.orchestra.run.smithi177.stdout:4998 write 0xb295e46 thru 0xb2a3a6b (0xdc26 bytes) 2024-05-08T03:20:54.459 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:54.457+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:54.463 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:54.463 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:54.463 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:54.463 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:54.463 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:54.473 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:54.475 INFO:teuthology.orchestra.run.smithi177.stdout:4999 read 0x127acd1 thru 0x127bd1b (0x104b bytes) 2024-05-08T03:20:54.476 INFO:teuthology.orchestra.run.smithi177.stdout:5000 read 0xa2499aa thru 0xa24eacd (0x5124 bytes) 2024-05-08T03:20:54.479 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.479 INFO:teuthology.orchestra.run.smithi177.stdout:5002 read 0xac4454 thru 0xad2f9f (0xeb4c bytes) 2024-05-08T03:20:54.479 INFO:teuthology.orchestra.run.smithi177.stdout:5003 punch from 0x5bf428b to 0x5c02bed, (0xe962 bytes) 2024-05-08T03:20:54.479 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.479 INFO:teuthology.orchestra.run.smithi177.stdout:5005 read 0xb2592fe thru 0xb267347 (0xe04a bytes) 2024-05-08T03:20:54.480 INFO:teuthology.orchestra.run.smithi177.stdout:5006 trunc from 0xc1dfe9f to 0xc769538 2024-05-08T03:20:54.482 INFO:teuthology.orchestra.run.smithi177.stdout:5007 writesame 0xaf533a3 thru 0xaf56b84 data_size 0x17(0x37e2 bytes) 2024-05-08T03:20:54.489 INFO:teuthology.orchestra.run.smithi177.stdout:5008 read 0x375e86f thru 0x376d09e (0xe830 bytes) 2024-05-08T03:20:54.489 INFO:teuthology.orchestra.run.smithi177.stdout:5010 writesame 0x310d575 thru 0x3113b7c data_size 0x2(0x6608 bytes) 2024-05-08T03:20:54.503 INFO:teuthology.orchestra.run.smithi177.stdout:5011 write 0x73a9 thru 0x762e (0x286 bytes) 2024-05-08T03:20:54.505 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.509 INFO:teuthology.orchestra.run.smithi177.stdout:5013 trunc from 0xc769538 to 0x9a4b85f 2024-05-08T03:20:54.524 INFO:teuthology.orchestra.run.smithi177.stdout:5014 punch from 0x9625cf7 to 0x962dc31, (0x7f3a bytes) 2024-05-08T03:20:54.536 INFO:teuthology.orchestra.run.smithi177.stdout:5016 writesame 0xc9fded8 thru 0xca01304 data_size 0x13(0x342d bytes) 2024-05-08T03:20:54.544 INFO:teuthology.orchestra.run.smithi177.stdout:5017 writesame 0xcca6c61 thru 0xccb0402 data_size 0xd(0x97a2 bytes) 2024-05-08T03:20:54.560 INFO:teuthology.orchestra.run.smithi177.stdout:5019 trunc from 0xccb0403 to 0x11c9c14 2024-05-08T03:20:54.560 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:54.559+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:54.593 INFO:teuthology.orchestra.run.smithi177.stdout:5020 trunc from 0x11c9c14 to 0xb065870 2024-05-08T03:20:54.600 INFO:teuthology.orchestra.run.smithi177.stdout:5023 read 0x9faf848 thru 0x9fb4d33 (0x54ec bytes) 2024-05-08T03:20:54.612 INFO:teuthology.orchestra.run.smithi177.stdout:5024 trunc from 0xb065870 to 0x244ffa2 2024-05-08T03:20:54.618 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.625 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:54.625 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:54.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:54.625 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:54.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.625 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.423410845999996, 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.629 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.423401396000003, 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.630 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:20:54.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.423388660000001, 2024-05-08T03:20:54.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:20:54.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.423378084000007, 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.639 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:54.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:20:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.423368697000001, 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:20:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.423359516999994, 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192534481999999, 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192502245, 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.657 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:20:54.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:20:54.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.660 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192488914999998, 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.661 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:20:54.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:20:54.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:20:54.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:20:54.666 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.666 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.666 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.667 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.667 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:54.667 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192473941000003, 2024-05-08T03:20:54.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:20:54.667 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.668 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.668 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.668 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:20:54.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:20:54.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:20:54.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:20:54.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:20:54.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:20:54.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:20:54.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.676 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.676 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192462632000002, 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.677 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.678 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.678 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:20:54.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.678 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:54.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:20:54.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:20:54.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:20:54.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:20:54.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:20:54.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:20:54.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:20:54.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:20:54.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:20:54.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:20:54.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.685 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.686 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.686 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:54.686 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192444963, 2024-05-08T03:20:54.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:20:54.687 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.687 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.687 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:20:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:20:54.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:20:54.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:20:54.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:20:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:20:54.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:20:54.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:20:54.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.694 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192429765, 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.695 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:20:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.699 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 86.192415671000006, 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.417461697999997, 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:20:54.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:20:54.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:20:54.709 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 77.494567896000007, 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:20:54.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:54.714 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 77.494554407999999, 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 77.494525426999999, 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:20:54.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 77.494498820999993, 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.723 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 77.061750559999993, 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:20:54.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:20:54.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:54.733 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:54.734 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:54.734 INFO:teuthology.orchestra.run.smithi177.stdout:5027 writesame 0x5114021 thru 0x51169e4 data_size 0x36(0x29c4 bytes) 2024-05-08T03:20:54.735 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5028 writesame 0xbaa2d3b thru 0xbaa41c6 data_size 0xa(0x148c bytes) 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5029 read 0x911a57f thru 0x9126515 (0xbf97 bytes) 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5031 trunc from 0xbaa41c7 to 0x1eeba3a 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5033 trunc from 0x1eeba3a to 0xac21bc0 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5034 write 0x26e668d thru 0x26f30b0 (0xca24 bytes) 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5035 trunc from 0xac21bc0 to 0x251c268 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5036 write 0x1e8a7a4 thru 0x1e8fe09 (0x5666 bytes) 2024-05-08T03:20:54.737 INFO:teuthology.orchestra.run.smithi177.stdout:5037 trunc from 0x251c268 to 0x12db082 2024-05-08T03:20:54.738 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.738 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:54.733+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:54.739 INFO:teuthology.orchestra.run.smithi177.stdout:5039 trunc from 0x12db082 to 0x5575437 2024-05-08T03:20:54.745 INFO:teuthology.orchestra.run.smithi177.stdout:5040 writesame 0x4817437 thru 0x481f051 data_size 0x1(0x7c1b bytes) 2024-05-08T03:20:54.757 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.764 INFO:teuthology.orchestra.run.smithi177.stdout:5044 trunc from 0x5575437 to 0xb384848 2024-05-08T03:20:54.780 INFO:teuthology.orchestra.run.smithi177.stdout:5045 writesame 0xe0f8a9f thru 0xe0fd304 data_size 0x3(0x4866 bytes) 2024-05-08T03:20:54.780 INFO:teuthology.orchestra.run.smithi177.stdout:5046 read 0x68b0a78 thru 0x68bb386 (0xa90f bytes) 2024-05-08T03:20:54.780 INFO:teuthology.orchestra.run.smithi177.stdout:5047 writesame 0x589e616 thru 0x58a070f data_size 0x1(0x20fa bytes) 2024-05-08T03:20:54.784 INFO:teuthology.orchestra.run.smithi177.stdout:5049 write 0x5abab46 thru 0x5abf833 (0x4cee bytes) 2024-05-08T03:20:54.808 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:20:54.809 INFO:teuthology.orchestra.run.smithi177.stdout:5051 trunc from 0xe0fd305 to 0x54858ea 2024-05-08T03:20:54.831 INFO:teuthology.orchestra.run.smithi177.stdout:5052 read 0x184cffd thru 0x1859d52 (0xcd56 bytes) 2024-05-08T03:20:54.833 INFO:teuthology.orchestra.run.smithi177.stdout:5053 trunc from 0x54858ea to 0x41a35bf 2024-05-08T03:20:54.835 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:54.834+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:54.847 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.847 INFO:teuthology.orchestra.run.smithi177.stdout:5055 punch from 0x31ab871 to 0x31b051b, (0x4caa bytes) 2024-05-08T03:20:54.847 INFO:teuthology.orchestra.run.smithi177.stdout:5061 punch from 0xd25cef to 0xd30770, (0xaa81 bytes) 2024-05-08T03:20:54.847 INFO:teuthology.orchestra.run.smithi177.stdout:5062 writesame 0x75313a thru 0x758552 data_size 0x1(0x5419 bytes) 2024-05-08T03:20:54.865 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:54.865 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:54.865 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:54.865 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:54.871 INFO:teuthology.orchestra.run.smithi177.stdout:5063 writesame 0xee0c5ac thru 0xee1b16a data_size 0x1(0xebbf bytes) 2024-05-08T03:20:54.871 INFO:teuthology.orchestra.run.smithi177.stdout:5064 read 0x66ee95b thru 0x66fb819 (0xcebf bytes) 2024-05-08T03:20:54.871 INFO:teuthology.orchestra.run.smithi177.stdout:5066 punch from 0xc1a1f59 to 0xc1a5eaf, (0x3f56 bytes) 2024-05-08T03:20:54.871 INFO:teuthology.orchestra.run.smithi177.stdout:5067 write 0x853ccac thru 0x8549c36 (0xcf8b bytes) 2024-05-08T03:20:54.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:54.883 INFO:teuthology.orchestra.run.smithi177.stdout:5068 write 0xe9281cb thru 0xe9305f7 (0x842d bytes) 2024-05-08T03:20:54.897 INFO:teuthology.orchestra.run.smithi177.stdout:5069 write 0xb6ed9b3 thru 0xb6f4aa8 (0x70f6 bytes) 2024-05-08T03:20:54.909 INFO:teuthology.orchestra.run.smithi177.stdout:5070 write 0xa2dc3b2 thru 0xa2e455d (0x81ac bytes) 2024-05-08T03:20:54.921 INFO:teuthology.orchestra.run.smithi177.stdout:5072 writesame 0x507cecb thru 0x508cc06 data_size 0x13(0xfd3c bytes) 2024-05-08T03:20:54.929 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:54.929 INFO:teuthology.orchestra.run.smithi177.stdout:5074 read 0xa0473b1 thru 0xa050b90 (0x97e0 bytes) 2024-05-08T03:20:54.930 INFO:teuthology.orchestra.run.smithi177.stdout:5075 write 0x8cb46a thru 0x8d26d8 (0x726f bytes) 2024-05-08T03:20:54.936 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:54.934+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:54.946 INFO:teuthology.orchestra.run.smithi177.stdout:5076 trunc from 0xee1b16b to 0x4f06660 2024-05-08T03:20:54.972 INFO:teuthology.orchestra.run.smithi177.stdout:5077 write 0x7f1d582 thru 0x7f2594a (0x83c9 bytes) 2024-05-08T03:20:54.993 INFO:teuthology.orchestra.run.smithi177.stdout:5079 write 0xd78dd4d thru 0xd79cb79 (0xee2d bytes) 2024-05-08T03:20:55.007 INFO:teuthology.orchestra.run.smithi177.stdout:5080 punch from 0xa660e2b to 0xa66b4a4, (0xa679 bytes) 2024-05-08T03:20:55.007 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.007 INFO:teuthology.orchestra.run.smithi177.stdout:5083 read 0x979091d thru 0x9794d1c (0x4400 bytes) 2024-05-08T03:20:55.007 INFO:teuthology.orchestra.run.smithi177.stdout:5084 trunc from 0xd79cb7a to 0xd6ac0f0 2024-05-08T03:20:55.015 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:55.015 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:55.015 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:55.015 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:55.015 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:55.021 INFO:teuthology.orchestra.run.smithi177.stdout:5085 writesame 0xceb8c24 thru 0xceb944b data_size 0x1d(0x828 bytes) 2024-05-08T03:20:55.025 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:55.027 INFO:teuthology.orchestra.run.smithi177.stdout:5086 writesame 0xb1a384 thru 0xb2742c data_size 0xd(0xd0a9 bytes) 2024-05-08T03:20:55.037 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.038 INFO:teuthology.orchestra.run.smithi177.stdout:5090 write 0xb5c4577 thru 0xb5caddb (0x6865 bytes) 2024-05-08T03:20:55.038 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:55.036+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:55.048 INFO:teuthology.orchestra.run.smithi177.stdout:5091 write 0x1d49571 thru 0x1d5462f (0xb0bf bytes) 2024-05-08T03:20:55.056 INFO:teuthology.orchestra.run.smithi177.stdout:5092 writesame 0x36ee69c thru 0x36f3846 data_size 0x45(0x51ab bytes) 2024-05-08T03:20:55.066 INFO:teuthology.orchestra.run.smithi177.stdout:5093 write 0xca01347 thru 0xca078d3 (0x658d bytes) 2024-05-08T03:20:55.077 INFO:teuthology.orchestra.run.smithi177.stdout:5094 write 0xcbd4506 thru 0xcbd9026 (0x4b21 bytes) 2024-05-08T03:20:55.094 INFO:teuthology.orchestra.run.smithi177.stdout:5095 trunc from 0xd6ac0f0 to 0x6cf165e 2024-05-08T03:20:55.119 INFO:teuthology.orchestra.run.smithi177.stdout:5097 read 0x4302b93 thru 0x4307a1d (0x4e8b bytes) 2024-05-08T03:20:55.119 INFO:teuthology.orchestra.run.smithi177.stdout:5098 write 0x35ca5d1 thru 0x35d9d66 (0xf796 bytes) 2024-05-08T03:20:55.133 INFO:teuthology.orchestra.run.smithi177.stdout:5099 read 0x53451eb thru 0x534802b (0x2e41 bytes) 2024-05-08T03:20:55.133 INFO:teuthology.orchestra.run.smithi177.stdout:5102 read 0x560c1ee thru 0x560c47f (0x292 bytes) 2024-05-08T03:20:55.133 INFO:teuthology.orchestra.run.smithi177.stdout:5103 read 0x29e446c thru 0x29e4a62 (0x5f7 bytes) 2024-05-08T03:20:55.133 INFO:teuthology.orchestra.run.smithi177.stdout:5104 read 0x542b80b thru 0x543857c (0xcd72 bytes) 2024-05-08T03:20:55.133 INFO:teuthology.orchestra.run.smithi177.stdout:5105 read 0x518321c thru 0x51922c3 (0xf0a8 bytes) 2024-05-08T03:20:55.134 INFO:teuthology.orchestra.run.smithi177.stdout:5106 read 0x2b40dac thru 0x2b4f0dd (0xe332 bytes) 2024-05-08T03:20:55.134 INFO:teuthology.orchestra.run.smithi177.stdout:5107 write 0xafd5d1 thru 0xb09937 (0xc367 bytes) 2024-05-08T03:20:55.137 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:55.136+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:55.138 INFO:teuthology.orchestra.run.smithi177.stdout:5109 read 0x12b4ae thru 0x12ef13 (0x3a66 bytes) 2024-05-08T03:20:55.142 INFO:teuthology.orchestra.run.smithi177.stdout:5110 trunc from 0x6cf165e to 0x9a506a8 2024-05-08T03:20:55.157 INFO:teuthology.orchestra.run.smithi177.stdout:5111 write 0xc4d1908 thru 0xc4d6292 (0x498b bytes) 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:55.160 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.225071713000006, 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:55.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:20:55.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3648951829999998, 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:20:55.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:20:55.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:20:55.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3648691580000003, 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:20:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:20:55.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:20:55.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3648271430000003, 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.178 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:20:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:20:55.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.183 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3638444669999998, 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:20:55.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:20:55.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.363808551, 2024-05-08T03:20:55.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:20:55.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:20:55.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:20:55.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3637835090000001, 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.193 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3613735890000003, 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.198 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:20:55.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:20:55.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3613492950000001, 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:20:55.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:20:55.207 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3612761779999998, 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:20:55.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:20:55.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:20:55.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.212 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3612479449999997, 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:20:55.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:20:55.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3612171980000003, 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.217 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3611798390000001, 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.226 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.359578108, 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.227 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:20:55.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3595240869999996, 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:20:55.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:20:55.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.236 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3594913809999998, 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3594669890000004, 2024-05-08T03:20:55.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:20:55.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3594475680000002, 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:20:55.246 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3594362889999996, 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.251 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.255 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 4.3594165949999999, 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.256 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:55.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.260 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:55.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:55.263 INFO:teuthology.orchestra.run.smithi177.stdout:5112 trunc from 0xc4d6293 to 0x182ecd2 2024-05-08T03:20:55.263 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.263 INFO:teuthology.orchestra.run.smithi177.stdout:5116 read 0xbddd53 thru 0xbe347b (0x5729 bytes) 2024-05-08T03:20:55.263 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5121 trunc from 0x182ecd2 to 0x77bc4e9 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5122 write 0xb800b04 thru 0xb80fd66 (0xf263 bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5123 read 0x4e7b323 thru 0x4e7c67a (0x1358 bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5125 read 0x2ac3c5 thru 0x2b6c33 (0xa86f bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5126 writesame 0xacf5ef7 thru 0xacf6a19 data_size 0x1(0xb23 bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5127 trunc from 0xb80fd67 to 0x76939e0 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5128 write 0xacaae thru 0xadc1e (0x1171 bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5131 read 0x4331848 thru 0x433a61b (0x8dd4 bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5136 read 0x494af69 thru 0x4954563 (0x95fb bytes) 2024-05-08T03:20:55.264 INFO:teuthology.orchestra.run.smithi177.stdout:5137 read 0x2ec78e7 thru 0x2ed1184 (0x989e bytes) 2024-05-08T03:20:55.265 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:55.260+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:55.268 INFO:teuthology.orchestra.run.smithi177.stdout:5138 trunc from 0x76939e0 to 0xd4328be 2024-05-08T03:20:55.273 INFO:teuthology.orchestra.run.smithi177.stdout:5139 writesame 0x6d8c848 thru 0x6d95aab data_size 0x24(0x9264 bytes) 2024-05-08T03:20:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:5141 read 0x9c38b93 thru 0x9c3e16c (0x55da bytes) 2024-05-08T03:20:55.282 INFO:teuthology.orchestra.run.smithi177.stdout:5142 writesame 0x71effd0 thru 0x71fc34b data_size 0x2(0xc37c bytes) 2024-05-08T03:20:55.298 INFO:teuthology.orchestra.run.smithi177.stdout:5143 read 0x68b7d22 thru 0x68c77c7 (0xfaa6 bytes) 2024-05-08T03:20:55.299 INFO:teuthology.orchestra.run.smithi177.stdout:5145 write 0xaa5dff3 thru 0xaa67973 (0x9981 bytes) 2024-05-08T03:20:55.308 INFO:teuthology.orchestra.run.smithi177.stdout:5146 read 0xb405025 thru 0xb40771d (0x26f9 bytes) 2024-05-08T03:20:55.308 INFO:teuthology.orchestra.run.smithi177.stdout:5147 write 0xbd4df86 thru 0xbd5b86d (0xd8e8 bytes) 2024-05-08T03:20:55.319 INFO:teuthology.orchestra.run.smithi177.stdout:5148 writesame 0x26a2242 thru 0x26a2569 data_size 0x4(0x328 bytes) 2024-05-08T03:20:55.322 INFO:teuthology.orchestra.run.smithi177.stdout:5149 write 0x8c37d5b thru 0x8c448c9 (0xcb6f bytes) 2024-05-08T03:20:55.337 INFO:teuthology.orchestra.run.smithi177.stdout:5151 trunc from 0xd4328be to 0xdea6fc5 2024-05-08T03:20:55.343 INFO:teuthology.orchestra.run.smithi177.stdout:5152 write 0x469f972 thru 0x46a8275 (0x8904 bytes) 2024-05-08T03:20:55.352 INFO:teuthology.orchestra.run.smithi177.stdout:5153 write 0x228b813 thru 0x228c52b (0xd19 bytes) 2024-05-08T03:20:55.356 INFO:teuthology.orchestra.run.smithi177.stdout:5154 write 0x3f2703a thru 0x3f35b62 (0xeb29 bytes) 2024-05-08T03:20:55.362 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:55.361+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:55.369 INFO:teuthology.orchestra.run.smithi177.stdout:5156 read 0x57d69c1 thru 0x57e2686 (0xbcc6 bytes) 2024-05-08T03:20:55.369 INFO:teuthology.orchestra.run.smithi177.stdout:5157 writesame 0x30486ee thru 0x304b96d data_size 0x20(0x3280 bytes) 2024-05-08T03:20:55.377 INFO:teuthology.orchestra.run.smithi177.stdout:5158 write 0xb4d593f thru 0xb4d6c55 (0x1317 bytes) 2024-05-08T03:20:55.383 INFO:teuthology.orchestra.run.smithi177.stdout:5159 read 0xd4cd9b0 thru 0xd4db4d3 (0xdb24 bytes) 2024-05-08T03:20:55.387 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:55.387 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:55.387 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:55.387 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:55.393 INFO:teuthology.orchestra.run.smithi177.stdout:5160 trunc from 0xdea6fc5 to 0x55f545f 2024-05-08T03:20:55.401 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:55.429 INFO:teuthology.orchestra.run.smithi177.stdout:5162 writesame 0x12a80dd thru 0x12af5a6 data_size 0x63(0x74ca bytes) 2024-05-08T03:20:55.438 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.450 INFO:teuthology.orchestra.run.smithi177.stdout:5164 write 0x8b93710 thru 0x8ba0c31 (0xd522 bytes) 2024-05-08T03:20:55.464 INFO:teuthology.orchestra.run.smithi177.stdout:5165 write 0x39939df thru 0x39a2805 (0xee27 bytes) 2024-05-08T03:20:55.464 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:55.462+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:55.472 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.472 INFO:teuthology.orchestra.run.smithi177.stdout:5169 punch from 0x722d099 to 0x7234265, (0x71cc bytes) 2024-05-08T03:20:55.480 INFO:teuthology.orchestra.run.smithi177.stdout:5170 writesame 0xaf44dd9 thru 0xaf4c050 data_size 0x48(0x7278 bytes) 2024-05-08T03:20:55.481 INFO:teuthology.orchestra.run.smithi177.stdout:5172 punch from 0x70db281 to 0x70ea5fa, (0xf379 bytes) 2024-05-08T03:20:55.481 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.490 INFO:teuthology.orchestra.run.smithi177.stdout:5174 trunc from 0xaf4c051 to 0x465d3d9 2024-05-08T03:20:55.512 INFO:teuthology.orchestra.run.smithi177.stdout:5176 trunc from 0x465d3d9 to 0xd7dd92 2024-05-08T03:20:55.562 INFO:teuthology.orchestra.run.smithi177.stdout:5177 write 0x6930537 thru 0x6937614 (0x70de bytes) 2024-05-08T03:20:55.565 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:55.564+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:55.565 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:55.565 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:55.565 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:55.565 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:55.565 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:55.575 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:55.576 INFO:teuthology.orchestra.run.smithi177.stdout:5178 read 0x2096dc thru 0x21119a (0x7abf bytes) 2024-05-08T03:20:55.588 INFO:teuthology.orchestra.run.smithi177.stdout:5179 write 0x8b37030 thru 0x8b3a695 (0x3666 bytes) 2024-05-08T03:20:55.610 INFO:teuthology.orchestra.run.smithi177.stdout:5180 write 0xa070a0e thru 0xa0713be (0x9b1 bytes) 2024-05-08T03:20:55.621 INFO:teuthology.orchestra.run.smithi177.stdout:5183 trunc from 0xa0713bf to 0x4c00138 2024-05-08T03:20:55.638 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.645 INFO:teuthology.orchestra.run.smithi177.stdout:5187 write 0x6e0777a thru 0x6e13863 (0xc0ea bytes) 2024-05-08T03:20:55.659 INFO:teuthology.orchestra.run.smithi177.stdout:5188 read 0x383e7ae thru 0x384c331 (0xdb84 bytes) 2024-05-08T03:20:55.659 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.659 INFO:teuthology.orchestra.run.smithi177.stdout:5190 punch from 0x1091a5a to 0x109a387, (0x892d bytes) 2024-05-08T03:20:55.660 INFO:teuthology.orchestra.run.smithi177.stdout:5191 writesame 0x636cf9a thru 0x637317e data_size 0x1(0x61e5 bytes) 2024-05-08T03:20:55.666 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:55.665+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:55.675 INFO:teuthology.orchestra.run.smithi177.stdout:5192 read 0x4f99c32 thru 0x4f9deb4 (0x4283 bytes) 2024-05-08T03:20:55.675 INFO:teuthology.orchestra.run.smithi177.stdout:5193 read 0x6084e2c thru 0x609375e (0xe933 bytes) 2024-05-08T03:20:55.676 INFO:teuthology.orchestra.run.smithi177.stdout:5195 write 0x6deb5f2 thru 0x6df6ae0 (0xb4ef bytes) 2024-05-08T03:20:55.692 INFO:teuthology.orchestra.run.smithi177.stdout:5196 write 0x92523b9 thru 0x925298d (0x5d5 bytes) 2024-05-08T03:20:55.696 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:55.696 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:55.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:55.696 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:55.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.696 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.593965676000003, 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:55.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.701 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.592245206000001, 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.702 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:55.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:55.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:55.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:55.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.592211765000002, 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.708 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.709 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:55.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:55.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:55.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:55.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:55.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.718 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.718 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:55.718 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.590786026000004, 2024-05-08T03:20:55.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:55.718 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.718 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.719 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:55.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:55.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:55.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:55.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:55.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:55.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:55.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:55.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:55.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.727 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.590693412, 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.728 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:55.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.732 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.590182738000003, 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.734 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.738 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.590155230000001, 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.739 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.589748798999999, 2024-05-08T03:20:55.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.589293421000001, 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.748 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.750 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:55.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:55.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:55.753 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.753 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.754 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.754 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.754 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:55.754 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.588765174000002, 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.755 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:55.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:55.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:55.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:55.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:55.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.761 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:55.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:55.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:55.763 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.764 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.764 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.764 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.764 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.588292676000002, 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.765 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.766 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.766 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:55.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.766 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:55.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:55.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:55.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:55.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:55.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:55.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.772 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.588214544000003, 2024-05-08T03:20:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:55.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:55.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.588139122000001, 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.785 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.588061959000001, 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:55.790 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.587984435999999, 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:55.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.587957523999997, 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.795 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:55.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:55.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:55.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.587936313999997, 2024-05-08T03:20:55.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:55.802 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.802 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:55.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:55.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:55.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:55.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:55.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:55.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:55.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:55.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:55.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:55.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.811 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:55.812 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.587916534000001, 2024-05-08T03:20:55.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:55.812 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.812 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.812 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:55.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:55.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:55.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:55.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:55.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:55.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:55.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.821 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.587362007999999, 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.822 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:55.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 47.587342995999997, 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:55.828 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:55.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:55.832 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:55.833 INFO:teuthology.orchestra.run.smithi177.stdout:5197 punch from 0x627a2db to 0x627da81, (0x37a6 bytes) 2024-05-08T03:20:55.833 INFO:teuthology.orchestra.run.smithi177.stdout:5198 trunc from 0x925298e to 0x458d587 2024-05-08T03:20:55.834 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:5202 trunc from 0x458d587 to 0x486c462 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:5203 read 0x134e96d thru 0x134f4ac (0xb40 bytes) 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:5204 read 0x2436dfe thru 0x243e0f4 (0x72f7 bytes) 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:5205 write 0x175724e thru 0x176685c (0xf60f bytes) 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:5206 writesame 0xa1c937b thru 0xa1ccdd0 data_size 0x39(0x3a56 bytes) 2024-05-08T03:20:55.837 INFO:teuthology.orchestra.run.smithi177.stdout:5207 read 0x3a6dfd2 thru 0x3a70487 (0x24b6 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5210 read 0x36a0382 thru 0x36a14e1 (0x1160 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5212 write 0x166f4d5 thru 0x167ece1 (0xf80d bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5213 punch from 0x956a077 to 0x9575329, (0xb2b2 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5215 read 0x44ffd61 thru 0x450842f (0x86cf bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5216 trunc from 0xa1ccdd1 to 0x9b90448 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5218 writesame 0x25a679 thru 0x262d0d data_size 0x1(0x8695 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5219 read 0x651fca thru 0x65bf29 (0x9f60 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5222 trunc from 0x9b90448 to 0xa40c036 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5223 trunc from 0xa40c036 to 0xbd08b7d 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5225 write 0x927e8dc thru 0x927f693 (0xdb8 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5226 writesame 0x9461fb4 thru 0x946cab5 data_size 0x35(0xab02 bytes) 2024-05-08T03:20:55.838 INFO:teuthology.orchestra.run.smithi177.stdout:5227 write 0x63c67ea thru 0x63d4c79 (0xe490 bytes) 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stdout:5228 write 0x2c23bdd thru 0x2c2583d (0x1c61 bytes) 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stdout:5229 read 0x1ed125e thru 0x1ed40c9 (0x2e6c bytes) 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stdout:5230 punch from 0x5ffa0b8 to 0x5ffbe30, (0x1d78 bytes) 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stdout:5231 trunc from 0xbd08b7d to 0x992af3b 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stdout:5233 punch from 0x77b447b to 0x77bc0e3, (0x7c68 bytes) 2024-05-08T03:20:55.839 INFO:teuthology.orchestra.run.smithi177.stdout:5235 write 0x3282e09 thru 0x3289944 (0x6b3c bytes) 2024-05-08T03:20:55.840 INFO:teuthology.orchestra.run.smithi177.stdout:5237 write 0x9eb44e3 thru 0x9ebf571 (0xb08f bytes) 2024-05-08T03:20:55.840 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:55.832+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:55.841 INFO:teuthology.orchestra.run.smithi177.stdout:5238 write 0x4060c35 thru 0x406ca14 (0xbde0 bytes) 2024-05-08T03:20:55.858 INFO:teuthology.orchestra.run.smithi177.stdout:5239 read 0x813654e thru 0x81382a6 (0x1d59 bytes) 2024-05-08T03:20:55.861 INFO:teuthology.orchestra.run.smithi177.stdout:5240 trunc from 0x9ebf572 to 0x7c02054 2024-05-08T03:20:55.889 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.889 INFO:teuthology.orchestra.run.smithi177.stdout:5242 punch from 0x393a2d2 to 0x393e0eb, (0x3e19 bytes) 2024-05-08T03:20:55.889 INFO:teuthology.orchestra.run.smithi177.stdout:5243 read 0x211f744 thru 0x211fbec (0x4a9 bytes) 2024-05-08T03:20:55.903 INFO:teuthology.orchestra.run.smithi177.stdout:5244 write 0xe5d8a8b thru 0xe5e2b95 (0xa10b bytes) 2024-05-08T03:20:55.916 INFO:teuthology.orchestra.run.smithi177.stdout:5247 write 0x85512b7 thru 0x8560080 (0xedca bytes) 2024-05-08T03:20:55.932 INFO:teuthology.orchestra.run.smithi177.stdout:5249 read 0x1804503 thru 0x1807a32 (0x3530 bytes) 2024-05-08T03:20:55.933 INFO:teuthology.orchestra.run.smithi177.stdout:5250 writesame 0x79347f4 thru 0x794244a data_size 0x1(0xdc57 bytes) 2024-05-08T03:20:55.934 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:55.933+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:55.947 INFO:teuthology.orchestra.run.smithi177.stdout:5252 punch from 0x557eb14 to 0x558562f, (0x6b1b bytes) 2024-05-08T03:20:55.947 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.947 INFO:teuthology.orchestra.run.smithi177.stdout:5254 read 0x4409c34 thru 0x4409e68 (0x235 bytes) 2024-05-08T03:20:55.947 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.947 INFO:teuthology.orchestra.run.smithi177.stdout:5256 read 0x5d372c3 thru 0x5d46bc6 (0xf904 bytes) 2024-05-08T03:20:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:55.960 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:55.960 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:55.960 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:55.961 INFO:teuthology.orchestra.run.smithi177.stdout:5257 trunc from 0xe5e2b96 to 0x281326c 2024-05-08T03:20:55.972 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:55.983 INFO:teuthology.orchestra.run.smithi177.stdout:5260 read 0x1b5bf26 thru 0x1b6a0c8 (0xe1a3 bytes) 2024-05-08T03:20:55.983 INFO:teuthology.orchestra.run.smithi177.stdout:5261 read 0x19f4505 thru 0x1a01ddb (0xd8d7 bytes) 2024-05-08T03:20:55.989 INFO:teuthology.orchestra.run.smithi177.stdout:5262 trunc from 0x281326c to 0x7486a1f 2024-05-08T03:20:55.996 INFO:teuthology.orchestra.run.smithi177.stdout:5263 read 0x176f0b6 thru 0x17776bf (0x860a bytes) 2024-05-08T03:20:55.996 INFO:teuthology.orchestra.run.smithi177.stdout:5265 read 0x6fd0c9 thru 0x703621 (0x6559 bytes) 2024-05-08T03:20:55.997 INFO:teuthology.orchestra.run.smithi177.stdout:5266 read 0x2e11d37 thru 0x2e15943 (0x3c0d bytes) 2024-05-08T03:20:55.997 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:55.997 INFO:teuthology.orchestra.run.smithi177.stdout:5268 read 0x2887511 thru 0x28896f7 (0x21e7 bytes) 2024-05-08T03:20:56.005 INFO:teuthology.orchestra.run.smithi177.stdout:5269 write 0x91be494 thru 0x91c767a (0x91e7 bytes) 2024-05-08T03:20:56.020 INFO:teuthology.orchestra.run.smithi177.stdout:5270 trunc from 0x91c767b to 0x41c0233 2024-05-08T03:20:56.035 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:56.033+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:56.052 INFO:teuthology.orchestra.run.smithi177.stdout:5271 write 0x9417ef8 thru 0x941fd1c (0x7e25 bytes) 2024-05-08T03:20:56.062 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.062 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.074 INFO:teuthology.orchestra.run.smithi177.stdout:5278 write 0xe053dd6 thru 0xe0547f5 (0xa20 bytes) 2024-05-08T03:20:56.084 INFO:teuthology.orchestra.run.smithi177.stdout:5280 writesame 0xab750 thru 0xb61ed data_size 0x1(0xaa9e bytes) 2024-05-08T03:20:56.092 INFO:teuthology.orchestra.run.smithi177.stdout:5281 write 0x4bab3f1 thru 0x4bb0318 (0x4f28 bytes) 2024-05-08T03:20:56.101 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.101 INFO:teuthology.orchestra.run.smithi177.stdout:5284 write 0x578f2f9 thru 0x579742a (0x8132 bytes) 2024-05-08T03:20:56.109 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:56.110 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:56.110 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:56.110 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:56.110 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:56.113 INFO:teuthology.orchestra.run.smithi177.stdout:5285 write 0x1126f12 thru 0x1136c1a (0xfd09 bytes) 2024-05-08T03:20:56.120 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:56.130 INFO:teuthology.orchestra.run.smithi177.stdout:5287 trunc from 0xe0547f6 to 0x627f325 2024-05-08T03:20:56.135 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:56.135+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:56.143 INFO:teuthology.orchestra.run.smithi177.stdout:5288 punch from 0x14dc502 to 0x14eb8ad, (0xf3ab bytes) 2024-05-08T03:20:56.144 INFO:teuthology.orchestra.run.smithi177.stdout:5290 read 0x10f8f4b thru 0x1103bb8 (0xac6e bytes) 2024-05-08T03:20:56.144 INFO:teuthology.orchestra.run.smithi177.stdout:5293 punch from 0x1f3de67 to 0x1f3f9d1, (0x1b6a bytes) 2024-05-08T03:20:56.144 INFO:teuthology.orchestra.run.smithi177.stdout:5294 write 0x5e975ba thru 0x5e97b48 (0x58f bytes) 2024-05-08T03:20:56.154 INFO:teuthology.orchestra.run.smithi177.stdout:5295 write 0x893eb39 thru 0x8940f06 (0x23ce bytes) 2024-05-08T03:20:56.159 INFO:teuthology.orchestra.run.smithi177.stdout:5296 write 0x6cf729a thru 0x6cfc1b9 (0x4f20 bytes) 2024-05-08T03:20:56.178 INFO:teuthology.orchestra.run.smithi177.stdout:5297 writesame 0xdb7e882 thru 0xdb869a7 data_size 0x3d(0x8126 bytes) 2024-05-08T03:20:56.178 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.178 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.178 INFO:teuthology.orchestra.run.smithi177.stdout:5301 read 0x19663c9 thru 0x1971726 (0xb35e bytes) 2024-05-08T03:20:56.179 INFO:teuthology.orchestra.run.smithi177.stdout:5302 punch from 0x8a8f4cb to 0x8a998ee, (0xa423 bytes) 2024-05-08T03:20:56.179 INFO:teuthology.orchestra.run.smithi177.stdout:5303 punch from 0x2e1dd5f to 0x2e24e3a, (0x70db bytes) 2024-05-08T03:20:56.179 INFO:teuthology.orchestra.run.smithi177.stdout:5304 read 0x1f069c4 thru 0x1f1273f (0xbd7c bytes) 2024-05-08T03:20:56.193 INFO:teuthology.orchestra.run.smithi177.stdout:5305 trunc from 0xdb869a8 to 0x2775f53 2024-05-08T03:20:56.208 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.208 INFO:teuthology.orchestra.run.smithi177.stdout:5307 read 0x528e69 thru 0x52dd2a (0x4ec2 bytes) 2024-05-08T03:20:56.227 INFO:teuthology.orchestra.run.smithi177.stdout:5309 write 0xe327b34 thru 0xe329cde (0x21ab bytes) 2024-05-08T03:20:56.237 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:56.235+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:56.246 INFO:teuthology.orchestra.run.smithi177.stdout:5310 trunc from 0xe329cdf to 0x17524b0 2024-05-08T03:20:56.262 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:56.262 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:56.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:56.262 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:56.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 57.263115849000002, 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 57.263048384999998, 2024-05-08T03:20:56.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:56.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:56.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:56.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:56.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 57.203214053000004, 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:56.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:56.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:56.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:56.282 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 56.13670604, 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:56.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:56.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:56.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 56.110168393999999, 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.288 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:56.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:56.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.294 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 56.082256626000003, 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:56.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:56.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 28.003676432999999, 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:20:56.300 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:56.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:56.305 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.999049057000001, 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:20:56.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:20:56.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.999028421999999, 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:20:56.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.999006171000001, 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.314 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:20:56.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:56.318 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.998985739999998, 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.322 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.998965513999998, 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.998944720000001, 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.327 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:20:56.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:20:56.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.998918706000001, 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.331 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.818961785999999, 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:20:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.818942681999999, 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:20:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:20:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.818381108000001, 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:20:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:20:56.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.813167246999999, 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:20:56.348 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 27.812999906000002, 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:20:56.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:20:56.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 19.450246309000001, 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:56.360 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:56.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:56.364 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:56.365 INFO:teuthology.orchestra.run.smithi177.stdout:5311 trunc from 0x17524b0 to 0x6c7ff39 2024-05-08T03:20:56.366 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:56.368 INFO:teuthology.orchestra.run.smithi177.stdout:5312 writesame 0xdf8cfa3 thru 0xdf906da data_size 0x4(0x3738 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5314 punch from 0x7f4232a to 0x7f4b033, (0x8d09 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5317 read 0x933bd12 thru 0x9348889 (0xcb78 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5318 writesame 0xe15e51c thru 0xe16191b data_size 0x40(0x3400 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5319 write 0x593a10f thru 0x5947b31 (0xda23 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5320 punch from 0x5370d50 to 0x537d331, (0xc5e1 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5324 read 0x43c5eaa thru 0x43ccd82 (0x6ed9 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5329 trunc from 0xe16191c to 0x71535b1 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5330 trunc from 0x71535b1 to 0xbd64ad1 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5333 punch from 0x59b08dc to 0x59bd2ab, (0xc9cf bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5334 read 0x62bee18 thru 0x62ca516 (0xb6ff bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5335 writesame 0x60f5e9d thru 0x6101d01 data_size 0x4d(0xbe65 bytes) 2024-05-08T03:20:56.369 INFO:teuthology.orchestra.run.smithi177.stdout:5336 punch from 0xa919e59 to 0xa9203e5, (0x658c bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5337 read 0x85b6be1 thru 0x85bd30e (0x672e bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5339 read 0x1523a93 thru 0x15296c2 (0x5c30 bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5341 punch from 0xafd6fff to 0xafd9eb2, (0x2eb3 bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5342 read 0x48c881 thru 0x48d6a6 (0xe26 bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5343 write 0x968255d thru 0x96922e2 (0xfd86 bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5345 read 0x28cbd20 thru 0x28cff92 (0x4273 bytes) 2024-05-08T03:20:56.370 INFO:teuthology.orchestra.run.smithi177.stdout:5346 trunc from 0xbd64ad1 to 0x1183c65 2024-05-08T03:20:56.370 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:56.365+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:56.417 INFO:teuthology.orchestra.run.smithi177.stdout:5348 writesame 0x7b406fe thru 0x7b44901 data_size 0x34(0x4204 bytes) 2024-05-08T03:20:56.418 INFO:teuthology.orchestra.run.smithi177.stdout:5351 read 0x2291a99 thru 0x229fcf6 (0xe25e bytes) 2024-05-08T03:20:56.418 INFO:teuthology.orchestra.run.smithi177.stdout:5352 read 0x65524fb thru 0x655a96a (0x8470 bytes) 2024-05-08T03:20:56.432 INFO:teuthology.orchestra.run.smithi177.stdout:5353 write 0xd69db3a thru 0xd6aaf5b (0xd422 bytes) 2024-05-08T03:20:56.444 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:20:56.446 INFO:teuthology.orchestra.run.smithi177.stdout:5355 read 0xcf408b3 thru 0xcf47736 (0x6e84 bytes) 2024-05-08T03:20:56.447 INFO:teuthology.orchestra.run.smithi177.stdout:5356 write 0xd53c9fa thru 0xd54284f (0x5e56 bytes) 2024-05-08T03:20:56.451 INFO:teuthology.orchestra.run.smithi177.stdout:5357 punch from 0xa15f953 to 0xa16d49d, (0xdb4a bytes) 2024-05-08T03:20:56.466 INFO:teuthology.orchestra.run.smithi177.stdout:5360 trunc from 0xd6aaf5c to 0x1795d0d 2024-05-08T03:20:56.466 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:56.465+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:56.484 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:56.484 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:56.484 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:56.484 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:56.493 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:56.501 INFO:teuthology.orchestra.run.smithi177.stdout:5361 trunc from 0x1795d0d to 0xb23751f 2024-05-08T03:20:56.509 INFO:teuthology.orchestra.run.smithi177.stdout:5363 punch from 0xaf2531a to 0xaf2f2ba, (0x9fa0 bytes) 2024-05-08T03:20:56.509 INFO:teuthology.orchestra.run.smithi177.stdout:5364 punch from 0x3e5d173 to 0x3e603c2, (0x324f bytes) 2024-05-08T03:20:56.509 INFO:teuthology.orchestra.run.smithi177.stdout:5367 write 0x225108a thru 0x225e5f4 (0xd56b bytes) 2024-05-08T03:20:56.526 INFO:teuthology.orchestra.run.smithi177.stdout:5368 read 0x8e1d451 thru 0x8e27f69 (0xab19 bytes) 2024-05-08T03:20:56.526 INFO:teuthology.orchestra.run.smithi177.stdout:5369 writesame 0x89b7c53 thru 0x89bc809 data_size 0x3(0x4bb7 bytes) 2024-05-08T03:20:56.534 INFO:teuthology.orchestra.run.smithi177.stdout:5370 write 0xa338bfb thru 0xa340000 (0x7406 bytes) 2024-05-08T03:20:56.546 INFO:teuthology.orchestra.run.smithi177.stdout:5372 read 0x219d040 thru 0x21a9ce9 (0xccaa bytes) 2024-05-08T03:20:56.557 INFO:teuthology.orchestra.run.smithi177.stdout:5374 trunc from 0xb23751f to 0x2bb3f35 2024-05-08T03:20:56.567 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:56.566+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:56.579 INFO:teuthology.orchestra.run.smithi177.stdout:5375 read 0x1e99b4d thru 0x1e9c4f0 (0x29a4 bytes) 2024-05-08T03:20:56.589 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:56.589 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:56.594 INFO:teuthology.orchestra.run.smithi177.stdout:5376 write 0x9ac9676 thru 0x9acf3ea (0x5d75 bytes) 2024-05-08T03:20:56.601 INFO:teuthology.orchestra.run.smithi177.stdout:5377 writesame 0x48d2386 thru 0x48d2b28 data_size 0x1(0x7a3 bytes) 2024-05-08T03:20:56.608 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:56.608 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:56.610 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:56.610 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:56.613 INFO:teuthology.orchestra.run.smithi177.stdout:5378 write 0xc655856 thru 0xc65673c (0xee7 bytes) 2024-05-08T03:20:56.617 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.617 INFO:teuthology.orchestra.run.smithi177.stdout:5381 write 0x8d86a66 thru 0x8d8fa42 (0x8fdd bytes) 2024-05-08T03:20:56.618 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:56.625 INFO:teuthology.orchestra.run.smithi177.stdout:5382 write 0x35e6876 thru 0x35e7e99 (0x1624 bytes) 2024-05-08T03:20:56.628 INFO:teuthology.orchestra.run.smithi177.stdout:5383 write 0x6c22b9c thru 0x6c268a4 (0x3d09 bytes) 2024-05-08T03:20:56.631 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:56.631 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:56.633 INFO:teuthology.orchestra.run.smithi177.stdout:5384 punch from 0x1633bf to 0x163adb, (0x71c bytes) 2024-05-08T03:20:56.637 INFO:teuthology.orchestra.run.smithi177.stdout:5385 trunc from 0xc65673d to 0x8b67f30 2024-05-08T03:20:56.651 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:56.651 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:56.656 INFO:teuthology.orchestra.run.smithi177.stdout:5386 trunc from 0x8b67f30 to 0x4751691 2024-05-08T03:20:56.668 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.668 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:56.667+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:56.671 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:56.671 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:56.672 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:56.683 INFO:teuthology.orchestra.run.smithi177.stdout:5388 writesame 0xc6f9cb9 thru 0xc7070ae data_size 0xd(0xd3f6 bytes) 2024-05-08T03:20:56.684 INFO:teuthology.orchestra.run.smithi177.stdout:5390 read 0x6933d30 thru 0x693cbfa (0x8ecb bytes) 2024-05-08T03:20:56.690 INFO:teuthology.orchestra.run.smithi177.stdout:5391 trunc from 0xc7070af to 0x6ba1bd9 2024-05-08T03:20:56.691 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:56.691 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:56.696 INFO:teuthology.orchestra.run.smithi177.stdout:5393 write 0x48ca25d thru 0x48d5cae (0xba52 bytes) 2024-05-08T03:20:56.711 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:56.711 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:56.714 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:56.715 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:56.716 INFO:teuthology.orchestra.run.smithi177.stdout:5394 write 0x80bca36 thru 0x80cc6a5 (0xfc70 bytes) 2024-05-08T03:20:56.735 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.735 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.735 INFO:teuthology.orchestra.run.smithi177.stdout:5398 read 0x7783058 thru 0x77846e8 (0x1691 bytes) 2024-05-08T03:20:56.735 INFO:teuthology.orchestra.run.smithi177.stdout:5399 punch from 0x50baf3f to 0x50c970e, (0xe7cf bytes) 2024-05-08T03:20:56.735 INFO:teuthology.orchestra.run.smithi177.stdout:5401 write 0x58e576e thru 0x58ec7f0 (0x7083 bytes) 2024-05-08T03:20:56.749 INFO:teuthology.orchestra.run.smithi177.stdout:5402 write 0x73bc140 thru 0x73c6b9a (0xaa5b bytes) 2024-05-08T03:20:56.763 INFO:teuthology.orchestra.run.smithi177.stdout:5405 trunc from 0x80cc6a6 to 0xabd7790 2024-05-08T03:20:56.768 INFO:teuthology.orchestra.run.smithi177.stdout:5406 punch from 0x2ec9293 to 0x2ed1c2f, (0x899c bytes) 2024-05-08T03:20:56.768 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.768 INFO:teuthology.orchestra.run.smithi177.stdout:5408 read 0x785f63c thru 0x786822d (0x8bf2 bytes) 2024-05-08T03:20:56.769 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.769 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.769 INFO:teuthology.orchestra.run.smithi177.stdout:5411 write 0x9bfc407 thru 0x9c01a11 (0x560b bytes) 2024-05-08T03:20:56.769 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:56.767+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:56.775 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:56.775 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:56.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:20:56.788 INFO:teuthology.orchestra.run.smithi177.stdout:5413 trunc from 0xabd7790 to 0xc0a3b6 2024-05-08T03:20:56.832 INFO:teuthology.orchestra.run.smithi177.stdout:5414 write 0x3e61e62 thru 0x3e677a4 (0x5943 bytes) 2024-05-08T03:20:56.840 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.840 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.857 INFO:teuthology.orchestra.run.smithi177.stdout:5419 write 0xe118cdf thru 0xe11b938 (0x2c5a bytes) 2024-05-08T03:20:56.861 INFO:teuthology.orchestra.run.smithi177.stdout:5420 write 0x4e44e50 thru 0x4e4f84d (0xa9fe bytes) 2024-05-08T03:20:56.869 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:56.868+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:56.871 INFO:teuthology.orchestra.run.smithi177.stdout:5421 read 0x69b001f thru 0x69bb3b2 (0xb394 bytes) 2024-05-08T03:20:56.871 INFO:teuthology.orchestra.run.smithi177.stdout:5422 writesame 0xadcfb23 thru 0xadd6aa7 data_size 0x1(0x6f85 bytes) 2024-05-08T03:20:56.884 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.884 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.884 INFO:teuthology.orchestra.run.smithi177.stdout:5425 punch from 0xae2c99f to 0xae35544, (0x8ba5 bytes) 2024-05-08T03:20:56.884 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:20:56.884 INFO:teuthology.orchestra.run.smithi177.stdout:5430 read 0xa707fce thru 0xa70c6a0 (0x46d3 bytes) 2024-05-08T03:20:56.885 INFO:teuthology.orchestra.run.smithi177.stdout:5431 read 0x292206c thru 0x2926c9a (0x4c2f bytes) 2024-05-08T03:20:56.885 INFO:teuthology.orchestra.run.smithi177.stdout:5432 read 0xd6a4c57 thru 0xd6ac9a7 (0x7d51 bytes) 2024-05-08T03:20:56.885 INFO:teuthology.orchestra.run.smithi177.stdout:5433 write 0xb602d7 thru 0xb6a324 (0xa04e bytes) 2024-05-08T03:20:56.889 INFO:teuthology.orchestra.run.smithi177.stdout:5434 trunc from 0xe11b939 to 0xc55b074 2024-05-08T03:20:56.901 INFO:teuthology.orchestra.run.smithi177.stdout:5435 write 0x530580e thru 0x530ccd8 (0x74cb bytes) 2024-05-08T03:20:56.902 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:56.902 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:56.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:20:56.970 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:56.970+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:57.018 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.018 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.027 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:20:57.072 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:57.071+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:57.172 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.172 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.173 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:57.171+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:57.180 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:20:57.273 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:57.273+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:57.327 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.327 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.335 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:20:57.374 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:57.373+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:57.476 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:57.475+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:57.480 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.480 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:20:57.576 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:57.575+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:57.635 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.636 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.644 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:20:57.689 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:57.687+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:57.756 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.756 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.765 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:20:57.778 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:57.777+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:57.809 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T03:20:57.809 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-05-08T03:20:57.809 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T03:20:57.809 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T03:20:57.812 INFO:tasks.ceph.osd.5:Started 2024-05-08T03:20:57.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:20:57.841 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:20:57.879 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:57.878+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:57.912 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:57.912 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:no valid command found; 10 closest matches: 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:0 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:1 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:2 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:abort 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:assert 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:config diff 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:config diff get 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:config get 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:config help [] 2024-05-08T03:20:57.913 INFO:teuthology.orchestra.run.smithi188.stderr:config set ... 2024-05-08T03:20:57.914 INFO:teuthology.orchestra.run.smithi188.stderr:admin_socket: invalid command 2024-05-08T03:20:57.914 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T03:20:57.915 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-05-08T03:20:57.921 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:20:57.980 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:57.978+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:58.032 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:58.033 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:58.042 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:20:58.082 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:58.081+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:58.182 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:58.180+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:58.188 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:58.188 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:58.197 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:20:58.283 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:58.281+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:58.342 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:58.342 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:58.351 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:20:58.392 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:58.391+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:58.399 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:20:58.398+0000 7fbf34501640 -1 Falling back to public interface 2024-05-08T03:20:58.485 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:58.483+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:58.496 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:58.496 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:58.505 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:20:58.586 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:58.585+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:58.650 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:58.651 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:58.659 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:20:58.703 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:58.702+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:58.716 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:20:58.766 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:20:58.766 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:20:58.775 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:20:58.788 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:58.787+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:58.821 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:20:58.845 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:20:58.845 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:20:58.863 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:20:58.863 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:20:58.881 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:20:58.881 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:20:58.888 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:58.887+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.791028611999998, 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:20:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:20:58.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.789308142000003, 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:20:58.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.789274700999997, 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:20:58.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:20:58.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.787848961999998, 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:20:58.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.787756348000002, 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.910 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:20:58.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.787245673999998, 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.916 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:58.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:20:58.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:20:58.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:58.920 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.787218166000002, 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:20:58.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:58.924 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.786811735000001, 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:58.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:20:58.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.928 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.786356357000003, 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:20:58.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.932 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785828109999997, 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:20:58.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785355612000004, 2024-05-08T03:20:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:20:58.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:20:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:20:58.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785277479999998, 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.942 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:58.946 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785202058000003, 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:20:58.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:20:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:20:58.950 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785124895000003, 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:20:58.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785047372000001, 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.955 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:20:58.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.959 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.785020459999998, 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.784999249999998, 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.964 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:20:58.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.784979470000003, 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:20:58.968 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.972 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.784424944000001, 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.973 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:58.977 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 50.784405931999999, 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:58.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:20:58.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:58.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:58.982 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:58.983 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:20:58.983 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:20:58.984 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:20:58.984 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:20:58.985 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:20:58.985 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:20:58.985 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:20:58.985 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:20:58.985 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:20:58.986 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:20:58.986 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:20:58.986 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:20:59.024 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:20:59.024+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:20:59.090 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:59.089+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:59.129 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:59.130 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:59.130 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:20:59.130 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:59.138 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:20:59.191 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:20:59.191+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:20:59.243 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:59.243 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:20:59.243 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:20:59.244 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:20:59.244 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:59.252 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:20:59.294 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:59.293+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:59.394 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:20:59.393+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.402477210999997, 2024-05-08T03:20:59.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:20:59.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:20:59.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.402409747, 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.408 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:59.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:20:59.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:20:59.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.342575414999999, 2024-05-08T03:20:59.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:59.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:20:59.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:20:59.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:20:59.422 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.276067402000002, 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.423 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:20:59.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.249529756000001, 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.430 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:20:59.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.221617987999998, 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:20:59.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.143037795000001, 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:20:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:20:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:20:59.446 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138410419, 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:59.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:20:59.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138389784000001, 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.452 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:20:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:20:59.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138367533, 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.461 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138347102000001, 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:20:59.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:20:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138326876000001, 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:20:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138306082, 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 31.138280068, 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:20:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 30.958323148000002, 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:20:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 30.958304043999998, 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:20:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 30.957742469999999, 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:20:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:20:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 30.952528609000002, 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:20:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:20:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:20:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:20:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 30.952361268000001, 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 22.589607671, 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:20:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:20:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:20:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:20:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:20:59.513 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:20:59.514 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:20:59.515 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:59.513+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:59.615 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:20:59.614+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:20:59.631 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:59.631 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:20:59.631 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:59.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:20:59.725 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:20:59.724+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:20:59.787 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:59.787 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:20:59.788 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:20:59.788 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:20:59.788 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:20:59.796 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:20:59.837 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:20:59.836+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:20:59.918 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:20:59.917+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1675921200000001, 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:20:59.943 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:20:59.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:20:59.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1529689150000006, 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:20:59.948 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:20:59.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1521596360000004, 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:20:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:20:59.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.961 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1512203589999999, 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:20:59.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:20:59.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:20:59.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:59.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.967 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1495872889999994, 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:20:59.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:20:59.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1481487930000007, 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.973 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:20:59.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:20:59.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1481289050000001, 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.978 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:20:59.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:20:59.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.982 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1481049859999999, 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1471338059999994, 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1471221969999998, 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:20:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:20:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1471124790000005, 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:20:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:20:59.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:20:59.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:20:59.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1471020169999999, 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.001 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.002 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1445250710000003, 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.006 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:00.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1444948240000006, 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.011 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:00.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:00.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1444678069999998, 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.016 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:00.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:00.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:00.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:00.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1444488140000004, 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.023 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:00.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:00.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1444187550000002, 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.028 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1426342910000002, 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:00.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:00.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1426224680000008, 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.042 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.1426106110000003, 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.043 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:00.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:00.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.047 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.049 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:00.050 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:00.048+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:00.150 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:00.149+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:00.154 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:00.154 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:00.154 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:00.155 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.163 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:00.251 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:00.250+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:00.309 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:00.309 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:00.309 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:00.309 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:00.309 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.318 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:00.359 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:00.358+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:00.453 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:00.452+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:00.465 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:00.465 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:00.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:00.465 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:00.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 84.254711259000004, 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:00.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:00.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:00.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:00.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.335030742000001, 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.472 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:21:00.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.335006671000002, 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.476 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:21:00.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334980947999995, 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.481 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.485 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334949230999996, 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.486 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334924126999994, 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.490 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334899321999998, 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.494 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:00.495 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.496 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.497 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.498 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334870777999996, 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:00.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:00.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.502 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334848256000001, 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:00.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:00.506 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334817916000006, 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.507 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.334796734999998, 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:00.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:00.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.515 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 69.522154799000006, 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.516 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:00.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:00.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:00.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.333480078000001, 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:00.522 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:00.527 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.311276735, 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.528 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:00.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:00.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.309580617000002, 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.532 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:00.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:00.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:00.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.289875829000003, 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:00.536 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:00.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:00.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:00.542 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.289851869000003, 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:00.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:00.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:00.549 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.249260323999998, 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:00.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:00.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:00.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:00.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 60.236486427000003, 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.558 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:00.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.563 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 40.975747493, 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:00.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:00.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.570 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.573 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:00.573 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:00.571+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:00.673 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:00.672+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:00.678 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:00.678 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:00.678 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:00.678 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.687 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:00.774 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:00.773+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:00.792 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:00.793 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:00.793 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:00.793 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:00.793 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.801 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:00.840 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:21:00.875 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:00.874+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 89.032650583999995, 2024-05-08T03:21:00.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:00.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:00.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690709392000002, 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:00.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:00.919 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690693992999996, 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:00.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:00.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.923 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690662351, 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.924 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:00.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690649019000006, 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:00.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:00.933 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690639224999998, 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:00.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.937 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690629811999997, 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:00.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690620291000002, 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.942 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:00.943 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:00.944 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:00.945 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690610950000007, 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.946 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.947 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.948 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.949 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690601735000001, 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:00.950 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.951 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.952 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:00.953 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690592373000001, 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.954 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.955 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.956 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:00.957 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690582941000002, 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:00.958 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.959 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:00.960 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.961 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.962 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690573525000005, 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.963 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.964 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.965 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.966 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690564507000005, 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.967 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:00.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690555463999999, 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.971 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.973 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690543171000002, 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.975 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:00.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:00.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690533904000006, 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:00.979 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690520046000003, 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:00.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:00.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.987 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.690510752999998, 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.988 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:00.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:00.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:00.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 84.697496619999995, 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:00.992 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:00.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:00.996 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:00.997 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:00.999 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:00.997+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:00.999 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:01.099 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:01.098+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:01.108 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:01.108 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:01.108 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:01.108 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:01.117 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:01.124 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:01.125 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:01.142 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:01.142 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:01.161 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:01.162 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:01.177 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:01.178 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:01.195 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:01.196 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:01.196 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:21:01.200 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:01.199+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:01.213 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:01.214 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:01.231 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:01.231 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:01.231 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:01.231 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:01.234 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:21:01.235 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:21:01.240 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:01.325 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:01.324+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:01.385 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:01.386 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:01.393 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:01.435 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:01.434+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:01.503 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:01.502+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:01.539 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:01.540 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:01.548 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:01.604 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:01.603+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:01.653 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:01.653 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:01.662 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:01.705 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:01.704+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:01.768 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:01.768 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:01.776 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:01.821 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:01.821+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:01.827 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:21:01.870 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:01.869+0000 7fbf34501640 -1 osd.5 58 log_to_monitors true 2024-05-08T03:21:01.885 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:01.886 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:01.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:01.907 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:01.906+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:02.000 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.000 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.008 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:02.007+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:02.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:02.109 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:02.108+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:02.115 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.115 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.124 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:02.209 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:02.208+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:02.269 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.269 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.279 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:02.310 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:02.309+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:02.383 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.383 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.392 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:02.411 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:02.410+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:02.497 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.497 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:02.511 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:02.510+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:02.614 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:02.613+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:02.624 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.624 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.633 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:02.715 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:02.713+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:02.778 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.779 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.787 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:02.816 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:02.814+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:02.860 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:02.859+0000 7fbf282a6640 -1 osd.5 105 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T03:21:02.902 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:02.902 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:02.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:02.916 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:02.917 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:02.915+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:03.017 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:03.016+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:03.018 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:03.018 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:03.028 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:03.061 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:03.062 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:03.062 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:03.062 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:03.071 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T03:21:03.118 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:03.117+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:03.166 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:03.166 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:03.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:03.219 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:03.218+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:03.236 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:03.279 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:03.280 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:03.288 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:03.329 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:03.328+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:03.358 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:03.358 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:03.375 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:03.376 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:03.392 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:03.393 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:03.409 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:03.409 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:03.420 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:03.419+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:03.426 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:03.426 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:03.426 INFO:teuthology.orchestra.run.smithi177.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T03:21:03.434 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:03.434 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:03.443 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:03.444 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:03.444 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:03.460 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:03.460 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:03.463 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T03:21:03.464 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T03:21:03.521 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:03.520+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:03.548 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:03.548 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:03.548 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:03.548 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:03.548 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:03.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:03.641 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:03.640+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:03.703 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:03.703 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 91.827315561999995, 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.704 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:03.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:03.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:03.709 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485374370000002, 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.710 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485358970999997, 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:03.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485327329, 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:03.718 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:03.722 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485313997000006, 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.726 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485304202999998, 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.727 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:03.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:03.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485294789999998, 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.731 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:03.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:03.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485285269000002, 2024-05-08T03:21:03.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:03.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.739 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485275927999993, 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485266713000001, 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:03.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:03.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485257351000001, 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:03.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485247919000003, 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.752 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:03.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:03.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485238503000005, 2024-05-08T03:21:03.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:03.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.760 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485229485000005, 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:03.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485220441999999, 2024-05-08T03:21:03.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:03.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485208149000002, 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.768 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:03.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485198882000006, 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.772 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485185024000003, 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.776 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 90.485175730999998, 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:03.780 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:03.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:03.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.783 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.492161597999996, 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:03.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:03.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:03.788 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.788 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.788 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.788 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:03.788 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:03.788 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:03.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:03.790 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:03.788+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:03.830 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T03:21:03.841 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:21:03.846 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T03:21:03.890 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:03.889+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:03.936 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:03.937 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:03.937 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:03.937 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:03.945 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:04.029 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:04.028+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:04.091 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:04.091 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:04.091 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:04.091 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:04.091 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:04.091 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:04.089+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:04.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:04.192 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:04.191+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:04.247 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:04.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:04.247 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:04.255 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:04.299 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:04.298+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:04.362 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:04.362 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:04.362 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:04.362 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:04.371 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:04.393 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:04.392+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:04.495 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:04.494+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:04.515 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:04.515 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:04.515 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:04.515 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:04.515 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:04.523 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:04.595 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:04.594+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:04.630 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:04.630 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:04.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:04.630 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:04.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.428315462, 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:04.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:04.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:04.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.428306011999993, 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:04.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.428293276000005, 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:04.639 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:04.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.643 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.428282699999997, 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.428273313000005, 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:04.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.651 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.428264132999999, 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:04.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:04.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197439098000004, 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:04.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:04.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197406861000005, 2024-05-08T03:21:04.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.664 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197393531000003, 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:04.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:04.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197378556999993, 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:04.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:04.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197367248000006, 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:04.673 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.677 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197349579000004, 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:04.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197334381000005, 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.682 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:04.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:04.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.197320286999997, 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.686 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:04.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:04.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 88.422366314000001, 2024-05-08T03:21:04.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:04.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:04.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 87.499472511999997, 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.696 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 87.499459024000004, 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.700 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:04.704 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 87.499430043000004, 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:04.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 87.499403436999998, 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:04.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 87.066655175999998, 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:04.713 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:04.718 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:04.720 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:04.720 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:04.718+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:04.820 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:04.819+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:04.843 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:21:04.844 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T03:21:04.866 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-08T03:21:04.871 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:04.871 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:04.871 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:04.880 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:04.922 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:04.921+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:05.022 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:05.021+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:05.029 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:05.029 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:05.029 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:05.029 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:05.029 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:05.039 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:05.123 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:05.122+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.250865184000006, 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.186 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:05.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:05.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:05.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.390688654, 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.193 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:05.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:05.197 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.390662628999999, 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.198 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.202 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.390620613999999, 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.203 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:05.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:05.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:05.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.389637938, 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:05.212 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.389602022, 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:05.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.389576979999999, 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.217 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:05.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:05.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.221 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.387167059999999, 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.222 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:05.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:05.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:05.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.387142766, 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.227 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:05.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.231 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.387069649000001, 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:05.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:05.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.387041416000001, 2024-05-08T03:21:05.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:05.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:05.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.387010669, 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.241 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:05.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:05.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:05.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:05.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.38697331, 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:05.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.385371578999999, 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:05.255 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.385317558000001, 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:05.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:05.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:05.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.385284852, 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:05.260 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.38526046, 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.266 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.385241039, 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.270 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:05.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:05.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.38522976, 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.275 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 14.385210066000001, 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:05.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:05.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:05.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:05.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:05.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.285 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.285 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.285 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:05.285 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:05.285 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:05.286 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:05.287 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:05.284+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:05.387 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:05.386+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:05.433 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:05.433 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:05.433 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:05.433 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:05.442 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:05.465 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:05.528 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:05.527+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:05.548 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:05.549 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:05.549 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:05.549 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:05.549 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:05.558 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:05.599 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:05.598+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:05.608 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:05.608 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:05.626 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:05.626 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:05.643 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:05.643 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:05.660 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:05.660 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:05.677 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:05.678 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:05.690 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:05.689+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:05.695 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:05.695 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-05-08T03:21:05.706 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:05.706 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:05.713 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:05.713 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:05.715 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:05.731 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:05.732 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:05.791 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:05.790+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:05.838 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:05.838 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:05.848 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:05.848 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.6 to 1 (10000) 2024-05-08T03:21:05.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T03:21:05.892 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:05.891+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:05.992 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:05.991+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:06.009 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.009 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.019 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:06.103 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:06.102+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:06.166 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.166 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:06.219 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:06.218+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:06.292 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.292 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.294 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:06.293+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:06.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:06.396 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:06.394+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:06.445 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.445 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.454 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:06.498 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:06.497+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:06.560 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.560 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.569 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:06.598 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:06.597+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:06.699 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:06.698+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:06.715 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.716 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.724 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:06.808 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:06.807+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:06.850 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T03:21:06.866 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:21:06.868 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T03:21:06.868 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:21:06.870 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:06.870 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:06.880 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:06.910 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:06.908+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:07.001 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:06.999+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:07.025 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.025 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.034 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:07.102 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:07.099+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:07.140 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.140 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:07.205 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:07.206 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:21:07.218 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-05-08T03:21:06.852889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1638051909,"num_objects":2202,"num_object_clones":486,"num_object_copies":4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2202,"num_whiteouts":934,"num_read":161177,"num_read_kb":1062722,"num_write":18841,"num_write_kb":653010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":612652156,"num_keys_recovered":712,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15055,"ondisk_log_size":15055,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1176288,"kb_used_data":939104,"kb_used_omap":6,"kb_used_meta":237113,"kb_avail":376311072,"statfs":{"total":386547056640,"available":385342537728,"internally_reserved":0,"allocated":961642496,"data_stored":1356026985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6379,"internal_metadata":242804501},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021820"},"pg_stats":[{"pgid":"3.1f","version":"102'308","reported_seq":2676,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847148+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:21:05.847148+0000","last_peered":"2024-05-08T03:21:05.847148+0000","last_clean":"2024-05-08T03:21:05.847148+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:21:05.847148+0000","last_undegraded":"2024-05-08T03:21:05.847148+0000","last_fullsized":"2024-05-08T03:21:05.847148+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:34:40.308722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":57482866,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":2583,"num_read_kb":29467,"num_write":361,"num_write_kb":16814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"102'253","reported_seq":2163,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846855+0000","last_change":"2024-05-08T03:20:50.785288+0000","last_active":"2024-05-08T03:21:05.846855+0000","last_peered":"2024-05-08T03:21:05.846855+0000","last_clean":"2024-05-08T03:21:05.846855+0000","last_became_active":"2024-05-08T03:20:50.783719+0000","last_became_peered":"2024-05-08T03:20:50.783719+0000","last_unstale":"2024-05-08T03:21:05.846855+0000","last_undegraded":"2024-05-08T03:21:05.846855+0000","last_fullsized":"2024-05-08T03:21:05.846855+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:30.667915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":25568922,"num_objects":60,"num_object_clones":10,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":2061,"num_read_kb":4877,"num_write":242,"num_write_kb":2882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"102'427","reported_seq":6223,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847176+0000","last_change":"2024-05-08T03:20:39.774078+0000","last_active":"2024-05-08T03:21:05.847176+0000","last_peered":"2024-05-08T03:21:05.847176+0000","last_clean":"2024-05-08T03:21:05.847176+0000","last_became_active":"2024-05-08T03:20:39.773251+0000","last_became_peered":"2024-05-08T03:20:39.773251+0000","last_unstale":"2024-05-08T03:21:05.847176+0000","last_undegraded":"2024-05-08T03:21:05.847176+0000","last_fullsized":"2024-05-08T03:21:05.847176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:41.903360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":72227752,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":20,"num_read":6458,"num_read_kb":42676,"num_write":510,"num_write_kb":13768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"102'855","reported_seq":3719,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846953+0000","last_change":"2024-05-08T03:20:40.781096+0000","last_active":"2024-05-08T03:21:05.846953+0000","last_peered":"2024-05-08T03:21:05.846953+0000","last_clean":"2024-05-08T03:21:05.846953+0000","last_became_active":"2024-05-08T03:20:40.779478+0000","last_became_peered":"2024-05-08T03:20:40.779478+0000","last_unstale":"2024-05-08T03:21:05.846953+0000","last_undegraded":"2024-05-08T03:21:05.846953+0000","last_fullsized":"2024-05-08T03:21:05.846953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:35:12.407208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":59895925,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":25,"num_read":5397,"num_read_kb":51808,"num_write":1228,"num_write_kb":51102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"102'389","reported_seq":3276,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846751+0000","last_change":"2024-05-08T03:20:40.780143+0000","last_active":"2024-05-08T03:21:05.846751+0000","last_peered":"2024-05-08T03:21:05.846751+0000","last_clean":"2024-05-08T03:21:05.846751+0000","last_became_active":"2024-05-08T03:20:40.779325+0000","last_became_peered":"2024-05-08T03:20:40.779325+0000","last_unstale":"2024-05-08T03:21:05.846751+0000","last_undegraded":"2024-05-08T03:21:05.846751+0000","last_fullsized":"2024-05-08T03:21:05.846751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:06:09.516862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":42007006,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":29,"num_read":4179,"num_read_kb":21558,"num_write":474,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7457235,"num_keys_recovered":60,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"102'519","reported_seq":4660,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846745+0000","last_change":"2024-05-08T03:20:39.775128+0000","last_active":"2024-05-08T03:21:05.846745+0000","last_peered":"2024-05-08T03:21:05.846745+0000","last_clean":"2024-05-08T03:21:05.846745+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:21:05.846745+0000","last_undegraded":"2024-05-08T03:21:05.846745+0000","last_fullsized":"2024-05-08T03:21:05.846745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:10.781474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":56983009,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":4960,"num_read_kb":40773,"num_write":682,"num_write_kb":45135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"102'836","reported_seq":3166,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846626+0000","last_change":"2024-05-08T03:20:51.782224+0000","last_active":"2024-05-08T03:21:05.846626+0000","last_peered":"2024-05-08T03:21:05.846626+0000","last_clean":"2024-05-08T03:21:05.846626+0000","last_became_active":"2024-05-08T03:20:51.780774+0000","last_became_peered":"2024-05-08T03:20:51.780774+0000","last_unstale":"2024-05-08T03:21:05.846626+0000","last_undegraded":"2024-05-08T03:21:05.846626+0000","last_fullsized":"2024-05-08T03:21:05.846626+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:25:38.631242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":46308347,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":5168,"num_read_kb":13745,"num_write":1294,"num_write_kb":7162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"102'430","reported_seq":8031,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846330+0000","last_change":"2024-05-08T03:20:50.779322+0000","last_active":"2024-05-08T03:21:05.846330+0000","last_peered":"2024-05-08T03:21:05.846330+0000","last_clean":"2024-05-08T03:21:05.846330+0000","last_became_active":"2024-05-08T03:20:50.777845+0000","last_became_peered":"2024-05-08T03:20:50.777845+0000","last_unstale":"2024-05-08T03:21:05.846330+0000","last_undegraded":"2024-05-08T03:21:05.846330+0000","last_fullsized":"2024-05-08T03:21:05.846330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:09:46.302704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":77162609,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":9150,"num_read_kb":56262,"num_write":440,"num_write_kb":15047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":18070131,"num_keys_recovered":60,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"102'310","reported_seq":4237,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846975+0000","last_change":"2024-05-08T03:20:50.781739+0000","last_active":"2024-05-08T03:21:05.846975+0000","last_peered":"2024-05-08T03:21:05.846975+0000","last_clean":"2024-05-08T03:21:05.846975+0000","last_became_active":"2024-05-08T03:20:50.780815+0000","last_became_peered":"2024-05-08T03:20:50.780815+0000","last_unstale":"2024-05-08T03:21:05.846975+0000","last_undegraded":"2024-05-08T03:21:05.846975+0000","last_fullsized":"2024-05-08T03:21:05.846975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:33.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":28645509,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":4655,"num_read_kb":22133,"num_write":329,"num_write_kb":4442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"102'216","reported_seq":2931,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846865+0000","last_change":"2024-05-08T03:20:51.783981+0000","last_active":"2024-05-08T03:21:05.846865+0000","last_peered":"2024-05-08T03:21:05.846865+0000","last_clean":"2024-05-08T03:21:05.846865+0000","last_became_active":"2024-05-08T03:20:51.783155+0000","last_became_peered":"2024-05-08T03:20:51.783155+0000","last_unstale":"2024-05-08T03:21:05.846865+0000","last_undegraded":"2024-05-08T03:21:05.846865+0000","last_fullsized":"2024-05-08T03:21:05.846865+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:14:26.480562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":92867256,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":28,"num_read":2284,"num_read_kb":60857,"num_write":209,"num_write_kb":45785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":54103866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"102'284","reported_seq":6986,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847117+0000","last_change":"2024-05-08T03:20:50.785464+0000","last_active":"2024-05-08T03:21:05.847117+0000","last_peered":"2024-05-08T03:21:05.847117+0000","last_clean":"2024-05-08T03:21:05.847117+0000","last_became_active":"2024-05-08T03:20:50.783826+0000","last_became_peered":"2024-05-08T03:20:50.783826+0000","last_unstale":"2024-05-08T03:21:05.847117+0000","last_undegraded":"2024-05-08T03:21:05.847117+0000","last_fullsized":"2024-05-08T03:21:05.847117+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:28.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":51848355,"num_objects":80,"num_object_clones":19,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":6381,"num_read_kb":35916,"num_write":293,"num_write_kb":7378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6817760,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"102'666","reported_seq":4079,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847167+0000","last_change":"2024-05-08T03:20:37.789834+0000","last_active":"2024-05-08T03:21:05.847167+0000","last_peered":"2024-05-08T03:21:05.847167+0000","last_clean":"2024-05-08T03:21:05.847167+0000","last_became_active":"2024-05-08T03:20:37.788932+0000","last_became_peered":"2024-05-08T03:20:37.788932+0000","last_unstale":"2024-05-08T03:21:05.847167+0000","last_undegraded":"2024-05-08T03:21:05.847167+0000","last_fullsized":"2024-05-08T03:21:05.847167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:41:43.934162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":32187150,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":5517,"num_read_kb":18315,"num_write":884,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":2507487,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":36,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847055+0000","last_change":"2024-05-08T03:20:47.820453+0000","last_active":"2024-05-08T03:21:05.847055+0000","last_peered":"2024-05-08T03:21:05.847055+0000","last_clean":"2024-05-08T03:21:05.847055+0000","last_became_active":"2024-05-08T03:20:47.820180+0000","last_became_peered":"2024-05-08T03:20:47.820180+0000","last_unstale":"2024-05-08T03:21:05.847055+0000","last_undegraded":"2024-05-08T03:21:05.847055+0000","last_fullsized":"2024-05-08T03:21:05.847055+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:33.427981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"102'300","reported_seq":3469,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846629+0000","last_change":"2024-05-08T03:20:50.782204+0000","last_active":"2024-05-08T03:21:05.846629+0000","last_peered":"2024-05-08T03:21:05.846629+0000","last_clean":"2024-05-08T03:21:05.846629+0000","last_became_active":"2024-05-08T03:20:50.781137+0000","last_became_peered":"2024-05-08T03:20:50.781137+0000","last_unstale":"2024-05-08T03:21:05.846629+0000","last_undegraded":"2024-05-08T03:21:05.846629+0000","last_fullsized":"2024-05-08T03:21:05.846629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:57.224862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":16719558,"num_objects":69,"num_object_clones":15,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":2980,"num_read_kb":11310,"num_write":326,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847029+0000","last_change":"2024-05-08T03:20:47.821931+0000","last_active":"2024-05-08T03:21:05.847029+0000","last_peered":"2024-05-08T03:21:05.847029+0000","last_clean":"2024-05-08T03:21:05.847029+0000","last_became_active":"2024-05-08T03:20:47.821642+0000","last_became_peered":"2024-05-08T03:20:47.821642+0000","last_unstale":"2024-05-08T03:21:05.847029+0000","last_undegraded":"2024-05-08T03:21:05.847029+0000","last_fullsized":"2024-05-08T03:21:05.847029+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:42.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"102'641","reported_seq":3647,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846813+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:21:05.846813+0000","last_peered":"2024-05-08T03:21:05.846813+0000","last_clean":"2024-05-08T03:21:05.846813+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:21:05.846813+0000","last_undegraded":"2024-05-08T03:21:05.846813+0000","last_fullsized":"2024-05-08T03:21:05.846813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:31:27.309931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":25330964,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":5381,"num_read_kb":30370,"num_write":850,"num_write_kb":17599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847311+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:21:05.847311+0000","last_peered":"2024-05-08T03:21:05.847311+0000","last_clean":"2024-05-08T03:21:05.847311+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:21:05.847311+0000","last_undegraded":"2024-05-08T03:21:05.847311+0000","last_fullsized":"2024-05-08T03:21:05.847311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"102'244","reported_seq":5078,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847087+0000","last_change":"2024-05-08T03:20:50.784522+0000","last_active":"2024-05-08T03:21:05.847087+0000","last_peered":"2024-05-08T03:21:05.847087+0000","last_clean":"2024-05-08T03:21:05.847087+0000","last_became_active":"2024-05-08T03:20:50.783297+0000","last_became_peered":"2024-05-08T03:20:50.783297+0000","last_unstale":"2024-05-08T03:21:05.847087+0000","last_undegraded":"2024-05-08T03:21:05.847087+0000","last_fullsized":"2024-05-08T03:21:05.847087+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:48:46.349399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":35507923,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":29,"num_read":4315,"num_read_kb":29035,"num_write":244,"num_write_kb":9515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":13300321,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846911+0000","last_change":"2024-05-08T03:20:47.821636+0000","last_active":"2024-05-08T03:21:05.846911+0000","last_peered":"2024-05-08T03:21:05.846911+0000","last_clean":"2024-05-08T03:21:05.846911+0000","last_became_active":"2024-05-08T03:20:47.821335+0000","last_became_peered":"2024-05-08T03:20:47.821335+0000","last_unstale":"2024-05-08T03:21:05.846911+0000","last_undegraded":"2024-05-08T03:21:05.846911+0000","last_fullsized":"2024-05-08T03:21:05.846911+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:23.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"102'875","reported_seq":3524,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846952+0000","last_change":"2024-05-08T03:20:38.806868+0000","last_active":"2024-05-08T03:21:05.846952+0000","last_peered":"2024-05-08T03:21:05.846952+0000","last_clean":"2024-05-08T03:21:05.846952+0000","last_became_active":"2024-05-08T03:20:38.806015+0000","last_became_peered":"2024-05-08T03:20:38.806015+0000","last_unstale":"2024-05-08T03:21:05.846952+0000","last_undegraded":"2024-05-08T03:21:05.846952+0000","last_fullsized":"2024-05-08T03:21:05.846952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:43.731687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":41500008,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":5070,"num_read_kb":30347,"num_write":1331,"num_write_kb":33092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":44040868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"102'296","reported_seq":3541,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846780+0000","last_change":"2024-05-08T03:20:50.784287+0000","last_active":"2024-05-08T03:21:05.846780+0000","last_peered":"2024-05-08T03:21:05.846780+0000","last_clean":"2024-05-08T03:21:05.846780+0000","last_became_active":"2024-05-08T03:20:50.783285+0000","last_became_peered":"2024-05-08T03:20:50.783285+0000","last_unstale":"2024-05-08T03:21:05.846780+0000","last_undegraded":"2024-05-08T03:21:05.846780+0000","last_fullsized":"2024-05-08T03:21:05.846780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:10.077862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":24644639,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2673,"num_read_kb":17605,"num_write":333,"num_write_kb":20964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14979418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":339,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846632+0000","last_change":"2024-05-08T03:20:50.784319+0000","last_active":"2024-05-08T03:21:05.846632+0000","last_peered":"2024-05-08T03:21:05.846632+0000","last_clean":"2024-05-08T03:21:05.846632+0000","last_became_active":"2024-05-08T03:20:50.783795+0000","last_became_peered":"2024-05-08T03:20:50.783795+0000","last_unstale":"2024-05-08T03:21:05.846632+0000","last_undegraded":"2024-05-08T03:21:05.846632+0000","last_fullsized":"2024-05-08T03:21:05.846632+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:15.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":194,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846750+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:21:05.846750+0000","last_peered":"2024-05-08T03:21:05.846750+0000","last_clean":"2024-05-08T03:21:05.846750+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:21:05.846750+0000","last_undegraded":"2024-05-08T03:21:05.846750+0000","last_fullsized":"2024-05-08T03:21:05.846750+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"102'353","reported_seq":5587,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846727+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:21:05.846727+0000","last_peered":"2024-05-08T03:21:05.846727+0000","last_clean":"2024-05-08T03:21:05.846727+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:21:05.846727+0000","last_undegraded":"2024-05-08T03:21:05.846727+0000","last_fullsized":"2024-05-08T03:21:05.846727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:15.337510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":36508356,"num_objects":86,"num_object_clones":24,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":36,"num_read":5893,"num_read_kb":23777,"num_write":323,"num_write_kb":17267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":62,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846851+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:21:05.846851+0000","last_peered":"2024-05-08T03:21:05.846851+0000","last_clean":"2024-05-08T03:21:05.846851+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:21:05.846851+0000","last_undegraded":"2024-05-08T03:21:05.846851+0000","last_fullsized":"2024-05-08T03:21:05.846851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"102'672","reported_seq":7025,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846930+0000","last_change":"2024-05-08T03:20:40.781092+0000","last_active":"2024-05-08T03:21:05.846930+0000","last_peered":"2024-05-08T03:21:05.846930+0000","last_clean":"2024-05-08T03:21:05.846930+0000","last_became_active":"2024-05-08T03:20:40.780249+0000","last_became_peered":"2024-05-08T03:20:40.780249+0000","last_unstale":"2024-05-08T03:21:05.846930+0000","last_undegraded":"2024-05-08T03:21:05.846930+0000","last_fullsized":"2024-05-08T03:21:05.846930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:04.115331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":63787864,"num_objects":73,"num_object_clones":16,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":27,"num_read":9736,"num_read_kb":48213,"num_write":887,"num_write_kb":21174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":195,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846292+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:21:05.846292+0000","last_peered":"2024-05-08T03:21:05.846292+0000","last_clean":"2024-05-08T03:21:05.846292+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:21:05.846292+0000","last_undegraded":"2024-05-08T03:21:05.846292+0000","last_fullsized":"2024-05-08T03:21:05.846292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"102'734","reported_seq":8267,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846695+0000","last_change":"2024-05-08T03:20:52.800241+0000","last_active":"2024-05-08T03:21:05.846695+0000","last_peered":"2024-05-08T03:21:05.846695+0000","last_clean":"2024-05-08T03:21:05.846695+0000","last_became_active":"2024-05-08T03:20:52.799061+0000","last_became_peered":"2024-05-08T03:20:52.799061+0000","last_unstale":"2024-05-08T03:21:05.846695+0000","last_undegraded":"2024-05-08T03:21:05.846695+0000","last_fullsized":"2024-05-08T03:21:05.846695+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:01.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":55140535,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":30,"num_read":9742,"num_read_kb":49929,"num_write":1034,"num_write_kb":16814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846427+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:21:05.846427+0000","last_peered":"2024-05-08T03:21:05.846427+0000","last_clean":"2024-05-08T03:21:05.846427+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:21:05.846427+0000","last_undegraded":"2024-05-08T03:21:05.846427+0000","last_fullsized":"2024-05-08T03:21:05.846427+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"102'410","reported_seq":4076,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846502+0000","last_change":"2024-05-08T03:20:51.781338+0000","last_active":"2024-05-08T03:21:05.846502+0000","last_peered":"2024-05-08T03:21:05.846502+0000","last_clean":"2024-05-08T03:21:05.846502+0000","last_became_active":"2024-05-08T03:20:51.780777+0000","last_became_peered":"2024-05-08T03:20:51.780777+0000","last_unstale":"2024-05-08T03:21:05.846502+0000","last_undegraded":"2024-05-08T03:21:05.846502+0000","last_fullsized":"2024-05-08T03:21:05.846502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:12.266795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":24467273,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":5927,"num_read_kb":14851,"num_write":407,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5273218,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"102'335","reported_seq":3187,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847169+0000","last_change":"2024-05-08T03:20:37.784597+0000","last_active":"2024-05-08T03:21:05.847169+0000","last_peered":"2024-05-08T03:21:05.847169+0000","last_clean":"2024-05-08T03:21:05.847169+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:21:05.847169+0000","last_undegraded":"2024-05-08T03:21:05.847169+0000","last_fullsized":"2024-05-08T03:21:05.847169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:10.896127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":84386704,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":3331,"num_read_kb":18937,"num_write":390,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"102'449","reported_seq":1780,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847259+0000","last_change":"2024-05-08T03:20:39.776155+0000","last_active":"2024-05-08T03:21:05.847259+0000","last_peered":"2024-05-08T03:21:05.847259+0000","last_clean":"2024-05-08T03:21:05.847259+0000","last_became_active":"2024-05-08T03:20:39.775306+0000","last_became_peered":"2024-05-08T03:20:39.775306+0000","last_unstale":"2024-05-08T03:21:05.847259+0000","last_undegraded":"2024-05-08T03:21:05.847259+0000","last_fullsized":"2024-05-08T03:21:05.847259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:39.981943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":25395708,"num_objects":42,"num_object_clones":5,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":2772,"num_read_kb":10028,"num_write":474,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"102'345","reported_seq":5902,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846749+0000","last_change":"2024-05-08T03:20:50.784394+0000","last_active":"2024-05-08T03:21:05.846749+0000","last_peered":"2024-05-08T03:21:05.846749+0000","last_clean":"2024-05-08T03:21:05.846749+0000","last_became_active":"2024-05-08T03:20:50.783233+0000","last_became_peered":"2024-05-08T03:20:50.783233+0000","last_unstale":"2024-05-08T03:21:05.846749+0000","last_undegraded":"2024-05-08T03:21:05.846749+0000","last_fullsized":"2024-05-08T03:21:05.846749+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:38:58.996315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":65249650,"num_objects":84,"num_object_clones":21,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":32,"num_read":5409,"num_read_kb":53155,"num_write":400,"num_write_kb":18336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":35463562,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"102'667","reported_seq":4464,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846757+0000","last_change":"2024-05-08T03:20:52.803353+0000","last_active":"2024-05-08T03:21:05.846757+0000","last_peered":"2024-05-08T03:21:05.846757+0000","last_clean":"2024-05-08T03:21:05.846757+0000","last_became_active":"2024-05-08T03:20:52.802662+0000","last_became_peered":"2024-05-08T03:20:52.802662+0000","last_unstale":"2024-05-08T03:21:05.846757+0000","last_undegraded":"2024-05-08T03:21:05.846757+0000","last_fullsized":"2024-05-08T03:21:05.846757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:51.312283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":86524717,"num_objects":80,"num_object_clones":19,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":36,"num_read":6400,"num_read_kb":22013,"num_write":909,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"102'845","reported_seq":4103,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846942+0000","last_change":"2024-05-08T03:20:51.785227+0000","last_active":"2024-05-08T03:21:05.846942+0000","last_peered":"2024-05-08T03:21:05.846942+0000","last_clean":"2024-05-08T03:21:05.846942+0000","last_became_active":"2024-05-08T03:20:51.783022+0000","last_became_peered":"2024-05-08T03:20:51.783022+0000","last_unstale":"2024-05-08T03:21:05.846942+0000","last_undegraded":"2024-05-08T03:21:05.846942+0000","last_fullsized":"2024-05-08T03:21:05.846942+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:42:50.226983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":84249527,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":31,"num_read":7745,"num_read_kb":71002,"num_write":1188,"num_write_kb":48330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":48212116,"num_keys_recovered":59,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"102'375","reported_seq":3714,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847208+0000","last_change":"2024-05-08T03:20:40.780044+0000","last_active":"2024-05-08T03:21:05.847208+0000","last_peered":"2024-05-08T03:21:05.847208+0000","last_clean":"2024-05-08T03:21:05.847208+0000","last_became_active":"2024-05-08T03:20:40.779118+0000","last_became_peered":"2024-05-08T03:20:40.779118+0000","last_unstale":"2024-05-08T03:21:05.847208+0000","last_undegraded":"2024-05-08T03:21:05.847208+0000","last_fullsized":"2024-05-08T03:21:05.847208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:30.162952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":70256478,"num_objects":83,"num_object_clones":22,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":31,"num_read":3278,"num_read_kb":54569,"num_write":443,"num_write_kb":29246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":51071767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"102'517","reported_seq":3723,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847086+0000","last_change":"2024-05-08T03:20:38.807093+0000","last_active":"2024-05-08T03:21:05.847086+0000","last_peered":"2024-05-08T03:21:05.847086+0000","last_clean":"2024-05-08T03:21:05.847086+0000","last_became_active":"2024-05-08T03:20:38.805719+0000","last_became_peered":"2024-05-08T03:20:38.805719+0000","last_unstale":"2024-05-08T03:21:05.847086+0000","last_undegraded":"2024-05-08T03:21:05.847086+0000","last_fullsized":"2024-05-08T03:21:05.847086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:29.670320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":27563758,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":3937,"num_read_kb":14749,"num_write":709,"num_write_kb":13908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":10326037,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"102'350","reported_seq":3463,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846246+0000","last_change":"2024-05-08T03:20:51.783432+0000","last_active":"2024-05-08T03:21:05.846246+0000","last_peered":"2024-05-08T03:21:05.846246+0000","last_clean":"2024-05-08T03:21:05.846246+0000","last_became_active":"2024-05-08T03:20:51.782569+0000","last_became_peered":"2024-05-08T03:20:51.782569+0000","last_unstale":"2024-05-08T03:21:05.846246+0000","last_undegraded":"2024-05-08T03:21:05.846246+0000","last_fullsized":"2024-05-08T03:21:05.846246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:43.399296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":70819277,"num_objects":73,"num_object_clones":16,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":32,"num_read":3884,"num_read_kb":35445,"num_write":357,"num_write_kb":18281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"102'407","reported_seq":5573,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846773+0000","last_change":"2024-05-08T03:20:38.806539+0000","last_active":"2024-05-08T03:21:05.846773+0000","last_peered":"2024-05-08T03:21:05.846773+0000","last_clean":"2024-05-08T03:21:05.846773+0000","last_became_active":"2024-05-08T03:20:38.805730+0000","last_became_peered":"2024-05-08T03:20:38.805730+0000","last_unstale":"2024-05-08T03:21:05.846773+0000","last_undegraded":"2024-05-08T03:21:05.846773+0000","last_fullsized":"2024-05-08T03:21:05.846773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:50.563335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":66655824,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":30,"num_read":6704,"num_read_kb":63814,"num_write":412,"num_write_kb":54964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":60512583,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"102'291","reported_seq":1838,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846624+0000","last_change":"2024-05-08T03:20:47.825128+0000","last_active":"2024-05-08T03:21:05.846624+0000","last_peered":"2024-05-08T03:21:05.846624+0000","last_clean":"2024-05-08T03:21:05.846624+0000","last_became_active":"2024-05-08T03:20:47.824316+0000","last_became_peered":"2024-05-08T03:20:47.824316+0000","last_unstale":"2024-05-08T03:21:05.846624+0000","last_undegraded":"2024-05-08T03:21:05.846624+0000","last_fullsized":"2024-05-08T03:21:05.846624+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:04:34.168774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":57536744,"num_objects":65,"num_object_clones":15,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":3292,"num_read_kb":26832,"num_write":285,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"102'339","reported_seq":3984,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846975+0000","last_change":"2024-05-08T03:20:38.806543+0000","last_active":"2024-05-08T03:21:05.846975+0000","last_peered":"2024-05-08T03:21:05.846975+0000","last_clean":"2024-05-08T03:21:05.846975+0000","last_became_active":"2024-05-08T03:20:38.805711+0000","last_became_peered":"2024-05-08T03:20:38.805711+0000","last_unstale":"2024-05-08T03:21:05.846975+0000","last_undegraded":"2024-05-08T03:21:05.846975+0000","last_fullsized":"2024-05-08T03:21:05.846975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:32:01.986000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":32031328,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":37,"num_read":3799,"num_read_kb":38254,"num_write":434,"num_write_kb":23177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":40436240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1637461541,"num_objects":2200,"num_object_clones":486,"num_object_copies":4400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2200,"num_whiteouts":934,"num_read":161061,"num_read_kb":1062622,"num_write":18682,"num_write_kb":650594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4136,"num_bytes_recovered":610881052,"num_keys_recovered":712,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1196371968,"data_stored":1703877269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14938,"ondisk_log_size":14938,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542170,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":259888,"kb_used_data":184288,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94111952,"statfs":{"total":96636764160,"available":96370638848,"internally_reserved":0,"allocated":188710912,"data_stored":273100882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":0,"seq":455266533378,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388052,"kb_used_data":338816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":93983788,"statfs":{"total":96636764160,"available":96239398912,"internally_reserved":0,"allocated":346947584,"data_stored":478836932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542171,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212832,"kb_used_data":163604,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94159008,"statfs":{"total":96636764160,"available":96418824192,"internally_reserved":0,"allocated":167530496,"data_stored":243430470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315516,"kb_used_data":252396,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94056324,"statfs":{"total":96636764160,"available":96313675776,"internally_reserved":0,"allocated":258453504,"data_stored":360658701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542171,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":256589824,"data_stored":359463175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166248448,"data_stored":242819412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":135417856,"data_stored":199525057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":345665536,"data_stored":478225874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":186847232,"data_stored":271905356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:21:07.220 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:21:07.233 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:07.231+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:07.294 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.294 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:07.303 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:07.301+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:07.403 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:07.402+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:07.448 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.448 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:07.505 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:07.503+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:07.541 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:07.541 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:21:07.553 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-05-08T03:21:06.852889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1638051909,"num_objects":2202,"num_object_clones":486,"num_object_copies":4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2202,"num_whiteouts":934,"num_read":161177,"num_read_kb":1062722,"num_write":18841,"num_write_kb":653010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":612652156,"num_keys_recovered":712,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15055,"ondisk_log_size":15055,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1176288,"kb_used_data":939104,"kb_used_omap":6,"kb_used_meta":237113,"kb_avail":376311072,"statfs":{"total":386547056640,"available":385342537728,"internally_reserved":0,"allocated":961642496,"data_stored":1356026985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6379,"internal_metadata":242804501},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021820"},"pg_stats":[{"pgid":"3.1f","version":"102'308","reported_seq":2676,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847148+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:21:05.847148+0000","last_peered":"2024-05-08T03:21:05.847148+0000","last_clean":"2024-05-08T03:21:05.847148+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:21:05.847148+0000","last_undegraded":"2024-05-08T03:21:05.847148+0000","last_fullsized":"2024-05-08T03:21:05.847148+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:34:40.308722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":57482866,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":2583,"num_read_kb":29467,"num_write":361,"num_write_kb":16814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"102'253","reported_seq":2163,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846855+0000","last_change":"2024-05-08T03:20:50.785288+0000","last_active":"2024-05-08T03:21:05.846855+0000","last_peered":"2024-05-08T03:21:05.846855+0000","last_clean":"2024-05-08T03:21:05.846855+0000","last_became_active":"2024-05-08T03:20:50.783719+0000","last_became_peered":"2024-05-08T03:20:50.783719+0000","last_unstale":"2024-05-08T03:21:05.846855+0000","last_undegraded":"2024-05-08T03:21:05.846855+0000","last_fullsized":"2024-05-08T03:21:05.846855+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:30.667915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":25568922,"num_objects":60,"num_object_clones":10,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":2061,"num_read_kb":4877,"num_write":242,"num_write_kb":2882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"102'427","reported_seq":6223,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847176+0000","last_change":"2024-05-08T03:20:39.774078+0000","last_active":"2024-05-08T03:21:05.847176+0000","last_peered":"2024-05-08T03:21:05.847176+0000","last_clean":"2024-05-08T03:21:05.847176+0000","last_became_active":"2024-05-08T03:20:39.773251+0000","last_became_peered":"2024-05-08T03:20:39.773251+0000","last_unstale":"2024-05-08T03:21:05.847176+0000","last_undegraded":"2024-05-08T03:21:05.847176+0000","last_fullsized":"2024-05-08T03:21:05.847176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:41.903360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":72227752,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":20,"num_read":6458,"num_read_kb":42676,"num_write":510,"num_write_kb":13768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"102'855","reported_seq":3719,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846953+0000","last_change":"2024-05-08T03:20:40.781096+0000","last_active":"2024-05-08T03:21:05.846953+0000","last_peered":"2024-05-08T03:21:05.846953+0000","last_clean":"2024-05-08T03:21:05.846953+0000","last_became_active":"2024-05-08T03:20:40.779478+0000","last_became_peered":"2024-05-08T03:20:40.779478+0000","last_unstale":"2024-05-08T03:21:05.846953+0000","last_undegraded":"2024-05-08T03:21:05.846953+0000","last_fullsized":"2024-05-08T03:21:05.846953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:35:12.407208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":59895925,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":25,"num_read":5397,"num_read_kb":51808,"num_write":1228,"num_write_kb":51102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"102'389","reported_seq":3276,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846751+0000","last_change":"2024-05-08T03:20:40.780143+0000","last_active":"2024-05-08T03:21:05.846751+0000","last_peered":"2024-05-08T03:21:05.846751+0000","last_clean":"2024-05-08T03:21:05.846751+0000","last_became_active":"2024-05-08T03:20:40.779325+0000","last_became_peered":"2024-05-08T03:20:40.779325+0000","last_unstale":"2024-05-08T03:21:05.846751+0000","last_undegraded":"2024-05-08T03:21:05.846751+0000","last_fullsized":"2024-05-08T03:21:05.846751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:06:09.516862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":42007006,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":29,"num_read":4179,"num_read_kb":21558,"num_write":474,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7457235,"num_keys_recovered":60,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"102'519","reported_seq":4660,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846745+0000","last_change":"2024-05-08T03:20:39.775128+0000","last_active":"2024-05-08T03:21:05.846745+0000","last_peered":"2024-05-08T03:21:05.846745+0000","last_clean":"2024-05-08T03:21:05.846745+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:21:05.846745+0000","last_undegraded":"2024-05-08T03:21:05.846745+0000","last_fullsized":"2024-05-08T03:21:05.846745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:10.781474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":56983009,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":4960,"num_read_kb":40773,"num_write":682,"num_write_kb":45135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"102'836","reported_seq":3166,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846626+0000","last_change":"2024-05-08T03:20:51.782224+0000","last_active":"2024-05-08T03:21:05.846626+0000","last_peered":"2024-05-08T03:21:05.846626+0000","last_clean":"2024-05-08T03:21:05.846626+0000","last_became_active":"2024-05-08T03:20:51.780774+0000","last_became_peered":"2024-05-08T03:20:51.780774+0000","last_unstale":"2024-05-08T03:21:05.846626+0000","last_undegraded":"2024-05-08T03:21:05.846626+0000","last_fullsized":"2024-05-08T03:21:05.846626+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:25:38.631242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":46308347,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":5168,"num_read_kb":13745,"num_write":1294,"num_write_kb":7162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"102'430","reported_seq":8031,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846330+0000","last_change":"2024-05-08T03:20:50.779322+0000","last_active":"2024-05-08T03:21:05.846330+0000","last_peered":"2024-05-08T03:21:05.846330+0000","last_clean":"2024-05-08T03:21:05.846330+0000","last_became_active":"2024-05-08T03:20:50.777845+0000","last_became_peered":"2024-05-08T03:20:50.777845+0000","last_unstale":"2024-05-08T03:21:05.846330+0000","last_undegraded":"2024-05-08T03:21:05.846330+0000","last_fullsized":"2024-05-08T03:21:05.846330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:09:46.302704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":77162609,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":9150,"num_read_kb":56262,"num_write":440,"num_write_kb":15047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":18070131,"num_keys_recovered":60,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"102'310","reported_seq":4237,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846975+0000","last_change":"2024-05-08T03:20:50.781739+0000","last_active":"2024-05-08T03:21:05.846975+0000","last_peered":"2024-05-08T03:21:05.846975+0000","last_clean":"2024-05-08T03:21:05.846975+0000","last_became_active":"2024-05-08T03:20:50.780815+0000","last_became_peered":"2024-05-08T03:20:50.780815+0000","last_unstale":"2024-05-08T03:21:05.846975+0000","last_undegraded":"2024-05-08T03:21:05.846975+0000","last_fullsized":"2024-05-08T03:21:05.846975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:33.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":28645509,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":4655,"num_read_kb":22133,"num_write":329,"num_write_kb":4442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"102'216","reported_seq":2931,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846865+0000","last_change":"2024-05-08T03:20:51.783981+0000","last_active":"2024-05-08T03:21:05.846865+0000","last_peered":"2024-05-08T03:21:05.846865+0000","last_clean":"2024-05-08T03:21:05.846865+0000","last_became_active":"2024-05-08T03:20:51.783155+0000","last_became_peered":"2024-05-08T03:20:51.783155+0000","last_unstale":"2024-05-08T03:21:05.846865+0000","last_undegraded":"2024-05-08T03:21:05.846865+0000","last_fullsized":"2024-05-08T03:21:05.846865+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:14:26.480562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":92867256,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":28,"num_read":2284,"num_read_kb":60857,"num_write":209,"num_write_kb":45785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":54103866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"102'284","reported_seq":6986,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847117+0000","last_change":"2024-05-08T03:20:50.785464+0000","last_active":"2024-05-08T03:21:05.847117+0000","last_peered":"2024-05-08T03:21:05.847117+0000","last_clean":"2024-05-08T03:21:05.847117+0000","last_became_active":"2024-05-08T03:20:50.783826+0000","last_became_peered":"2024-05-08T03:20:50.783826+0000","last_unstale":"2024-05-08T03:21:05.847117+0000","last_undegraded":"2024-05-08T03:21:05.847117+0000","last_fullsized":"2024-05-08T03:21:05.847117+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:28.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":51848355,"num_objects":80,"num_object_clones":19,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":6381,"num_read_kb":35916,"num_write":293,"num_write_kb":7378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6817760,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"102'666","reported_seq":4079,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847167+0000","last_change":"2024-05-08T03:20:37.789834+0000","last_active":"2024-05-08T03:21:05.847167+0000","last_peered":"2024-05-08T03:21:05.847167+0000","last_clean":"2024-05-08T03:21:05.847167+0000","last_became_active":"2024-05-08T03:20:37.788932+0000","last_became_peered":"2024-05-08T03:20:37.788932+0000","last_unstale":"2024-05-08T03:21:05.847167+0000","last_undegraded":"2024-05-08T03:21:05.847167+0000","last_fullsized":"2024-05-08T03:21:05.847167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:41:43.934162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":32187150,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":5517,"num_read_kb":18315,"num_write":884,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":2507487,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":36,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847055+0000","last_change":"2024-05-08T03:20:47.820453+0000","last_active":"2024-05-08T03:21:05.847055+0000","last_peered":"2024-05-08T03:21:05.847055+0000","last_clean":"2024-05-08T03:21:05.847055+0000","last_became_active":"2024-05-08T03:20:47.820180+0000","last_became_peered":"2024-05-08T03:20:47.820180+0000","last_unstale":"2024-05-08T03:21:05.847055+0000","last_undegraded":"2024-05-08T03:21:05.847055+0000","last_fullsized":"2024-05-08T03:21:05.847055+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:33.427981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"102'300","reported_seq":3469,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846629+0000","last_change":"2024-05-08T03:20:50.782204+0000","last_active":"2024-05-08T03:21:05.846629+0000","last_peered":"2024-05-08T03:21:05.846629+0000","last_clean":"2024-05-08T03:21:05.846629+0000","last_became_active":"2024-05-08T03:20:50.781137+0000","last_became_peered":"2024-05-08T03:20:50.781137+0000","last_unstale":"2024-05-08T03:21:05.846629+0000","last_undegraded":"2024-05-08T03:21:05.846629+0000","last_fullsized":"2024-05-08T03:21:05.846629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:57.224862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":16719558,"num_objects":69,"num_object_clones":15,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":2980,"num_read_kb":11310,"num_write":326,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847029+0000","last_change":"2024-05-08T03:20:47.821931+0000","last_active":"2024-05-08T03:21:05.847029+0000","last_peered":"2024-05-08T03:21:05.847029+0000","last_clean":"2024-05-08T03:21:05.847029+0000","last_became_active":"2024-05-08T03:20:47.821642+0000","last_became_peered":"2024-05-08T03:20:47.821642+0000","last_unstale":"2024-05-08T03:21:05.847029+0000","last_undegraded":"2024-05-08T03:21:05.847029+0000","last_fullsized":"2024-05-08T03:21:05.847029+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:42.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"102'641","reported_seq":3647,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846813+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:21:05.846813+0000","last_peered":"2024-05-08T03:21:05.846813+0000","last_clean":"2024-05-08T03:21:05.846813+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:21:05.846813+0000","last_undegraded":"2024-05-08T03:21:05.846813+0000","last_fullsized":"2024-05-08T03:21:05.846813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:31:27.309931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":25330964,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":5381,"num_read_kb":30370,"num_write":850,"num_write_kb":17599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847311+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:21:05.847311+0000","last_peered":"2024-05-08T03:21:05.847311+0000","last_clean":"2024-05-08T03:21:05.847311+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:21:05.847311+0000","last_undegraded":"2024-05-08T03:21:05.847311+0000","last_fullsized":"2024-05-08T03:21:05.847311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"102'244","reported_seq":5078,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847087+0000","last_change":"2024-05-08T03:20:50.784522+0000","last_active":"2024-05-08T03:21:05.847087+0000","last_peered":"2024-05-08T03:21:05.847087+0000","last_clean":"2024-05-08T03:21:05.847087+0000","last_became_active":"2024-05-08T03:20:50.783297+0000","last_became_peered":"2024-05-08T03:20:50.783297+0000","last_unstale":"2024-05-08T03:21:05.847087+0000","last_undegraded":"2024-05-08T03:21:05.847087+0000","last_fullsized":"2024-05-08T03:21:05.847087+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:48:46.349399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":35507923,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":29,"num_read":4315,"num_read_kb":29035,"num_write":244,"num_write_kb":9515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":13300321,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846911+0000","last_change":"2024-05-08T03:20:47.821636+0000","last_active":"2024-05-08T03:21:05.846911+0000","last_peered":"2024-05-08T03:21:05.846911+0000","last_clean":"2024-05-08T03:21:05.846911+0000","last_became_active":"2024-05-08T03:20:47.821335+0000","last_became_peered":"2024-05-08T03:20:47.821335+0000","last_unstale":"2024-05-08T03:21:05.846911+0000","last_undegraded":"2024-05-08T03:21:05.846911+0000","last_fullsized":"2024-05-08T03:21:05.846911+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:23.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"102'875","reported_seq":3524,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846952+0000","last_change":"2024-05-08T03:20:38.806868+0000","last_active":"2024-05-08T03:21:05.846952+0000","last_peered":"2024-05-08T03:21:05.846952+0000","last_clean":"2024-05-08T03:21:05.846952+0000","last_became_active":"2024-05-08T03:20:38.806015+0000","last_became_peered":"2024-05-08T03:20:38.806015+0000","last_unstale":"2024-05-08T03:21:05.846952+0000","last_undegraded":"2024-05-08T03:21:05.846952+0000","last_fullsized":"2024-05-08T03:21:05.846952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:43.731687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":41500008,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":5070,"num_read_kb":30347,"num_write":1331,"num_write_kb":33092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":44040868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"102'296","reported_seq":3541,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846780+0000","last_change":"2024-05-08T03:20:50.784287+0000","last_active":"2024-05-08T03:21:05.846780+0000","last_peered":"2024-05-08T03:21:05.846780+0000","last_clean":"2024-05-08T03:21:05.846780+0000","last_became_active":"2024-05-08T03:20:50.783285+0000","last_became_peered":"2024-05-08T03:20:50.783285+0000","last_unstale":"2024-05-08T03:21:05.846780+0000","last_undegraded":"2024-05-08T03:21:05.846780+0000","last_fullsized":"2024-05-08T03:21:05.846780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:10.077862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":24644639,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2673,"num_read_kb":17605,"num_write":333,"num_write_kb":20964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14979418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":339,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846632+0000","last_change":"2024-05-08T03:20:50.784319+0000","last_active":"2024-05-08T03:21:05.846632+0000","last_peered":"2024-05-08T03:21:05.846632+0000","last_clean":"2024-05-08T03:21:05.846632+0000","last_became_active":"2024-05-08T03:20:50.783795+0000","last_became_peered":"2024-05-08T03:20:50.783795+0000","last_unstale":"2024-05-08T03:21:05.846632+0000","last_undegraded":"2024-05-08T03:21:05.846632+0000","last_fullsized":"2024-05-08T03:21:05.846632+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:15.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":194,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846750+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:21:05.846750+0000","last_peered":"2024-05-08T03:21:05.846750+0000","last_clean":"2024-05-08T03:21:05.846750+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:21:05.846750+0000","last_undegraded":"2024-05-08T03:21:05.846750+0000","last_fullsized":"2024-05-08T03:21:05.846750+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"102'353","reported_seq":5587,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846727+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:21:05.846727+0000","last_peered":"2024-05-08T03:21:05.846727+0000","last_clean":"2024-05-08T03:21:05.846727+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:21:05.846727+0000","last_undegraded":"2024-05-08T03:21:05.846727+0000","last_fullsized":"2024-05-08T03:21:05.846727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:15.337510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":36508356,"num_objects":86,"num_object_clones":24,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":36,"num_read":5893,"num_read_kb":23777,"num_write":323,"num_write_kb":17267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":62,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846851+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:21:05.846851+0000","last_peered":"2024-05-08T03:21:05.846851+0000","last_clean":"2024-05-08T03:21:05.846851+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:21:05.846851+0000","last_undegraded":"2024-05-08T03:21:05.846851+0000","last_fullsized":"2024-05-08T03:21:05.846851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"102'672","reported_seq":7025,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846930+0000","last_change":"2024-05-08T03:20:40.781092+0000","last_active":"2024-05-08T03:21:05.846930+0000","last_peered":"2024-05-08T03:21:05.846930+0000","last_clean":"2024-05-08T03:21:05.846930+0000","last_became_active":"2024-05-08T03:20:40.780249+0000","last_became_peered":"2024-05-08T03:20:40.780249+0000","last_unstale":"2024-05-08T03:21:05.846930+0000","last_undegraded":"2024-05-08T03:21:05.846930+0000","last_fullsized":"2024-05-08T03:21:05.846930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:04.115331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":63787864,"num_objects":73,"num_object_clones":16,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":27,"num_read":9736,"num_read_kb":48213,"num_write":887,"num_write_kb":21174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":195,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846292+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:21:05.846292+0000","last_peered":"2024-05-08T03:21:05.846292+0000","last_clean":"2024-05-08T03:21:05.846292+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:21:05.846292+0000","last_undegraded":"2024-05-08T03:21:05.846292+0000","last_fullsized":"2024-05-08T03:21:05.846292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"102'734","reported_seq":8267,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846695+0000","last_change":"2024-05-08T03:20:52.800241+0000","last_active":"2024-05-08T03:21:05.846695+0000","last_peered":"2024-05-08T03:21:05.846695+0000","last_clean":"2024-05-08T03:21:05.846695+0000","last_became_active":"2024-05-08T03:20:52.799061+0000","last_became_peered":"2024-05-08T03:20:52.799061+0000","last_unstale":"2024-05-08T03:21:05.846695+0000","last_undegraded":"2024-05-08T03:21:05.846695+0000","last_fullsized":"2024-05-08T03:21:05.846695+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:01.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":55140535,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":30,"num_read":9742,"num_read_kb":49929,"num_write":1034,"num_write_kb":16814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846427+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:21:05.846427+0000","last_peered":"2024-05-08T03:21:05.846427+0000","last_clean":"2024-05-08T03:21:05.846427+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:21:05.846427+0000","last_undegraded":"2024-05-08T03:21:05.846427+0000","last_fullsized":"2024-05-08T03:21:05.846427+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"102'410","reported_seq":4076,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846502+0000","last_change":"2024-05-08T03:20:51.781338+0000","last_active":"2024-05-08T03:21:05.846502+0000","last_peered":"2024-05-08T03:21:05.846502+0000","last_clean":"2024-05-08T03:21:05.846502+0000","last_became_active":"2024-05-08T03:20:51.780777+0000","last_became_peered":"2024-05-08T03:20:51.780777+0000","last_unstale":"2024-05-08T03:21:05.846502+0000","last_undegraded":"2024-05-08T03:21:05.846502+0000","last_fullsized":"2024-05-08T03:21:05.846502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:12.266795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":24467273,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":5927,"num_read_kb":14851,"num_write":407,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5273218,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"102'335","reported_seq":3187,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847169+0000","last_change":"2024-05-08T03:20:37.784597+0000","last_active":"2024-05-08T03:21:05.847169+0000","last_peered":"2024-05-08T03:21:05.847169+0000","last_clean":"2024-05-08T03:21:05.847169+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:21:05.847169+0000","last_undegraded":"2024-05-08T03:21:05.847169+0000","last_fullsized":"2024-05-08T03:21:05.847169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:10.896127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":84386704,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":3331,"num_read_kb":18937,"num_write":390,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"102'449","reported_seq":1780,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847259+0000","last_change":"2024-05-08T03:20:39.776155+0000","last_active":"2024-05-08T03:21:05.847259+0000","last_peered":"2024-05-08T03:21:05.847259+0000","last_clean":"2024-05-08T03:21:05.847259+0000","last_became_active":"2024-05-08T03:20:39.775306+0000","last_became_peered":"2024-05-08T03:20:39.775306+0000","last_unstale":"2024-05-08T03:21:05.847259+0000","last_undegraded":"2024-05-08T03:21:05.847259+0000","last_fullsized":"2024-05-08T03:21:05.847259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:39.981943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":25395708,"num_objects":42,"num_object_clones":5,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":2772,"num_read_kb":10028,"num_write":474,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"102'345","reported_seq":5902,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846749+0000","last_change":"2024-05-08T03:20:50.784394+0000","last_active":"2024-05-08T03:21:05.846749+0000","last_peered":"2024-05-08T03:21:05.846749+0000","last_clean":"2024-05-08T03:21:05.846749+0000","last_became_active":"2024-05-08T03:20:50.783233+0000","last_became_peered":"2024-05-08T03:20:50.783233+0000","last_unstale":"2024-05-08T03:21:05.846749+0000","last_undegraded":"2024-05-08T03:21:05.846749+0000","last_fullsized":"2024-05-08T03:21:05.846749+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:38:58.996315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":65249650,"num_objects":84,"num_object_clones":21,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":32,"num_read":5409,"num_read_kb":53155,"num_write":400,"num_write_kb":18336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":35463562,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"102'667","reported_seq":4464,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846757+0000","last_change":"2024-05-08T03:20:52.803353+0000","last_active":"2024-05-08T03:21:05.846757+0000","last_peered":"2024-05-08T03:21:05.846757+0000","last_clean":"2024-05-08T03:21:05.846757+0000","last_became_active":"2024-05-08T03:20:52.802662+0000","last_became_peered":"2024-05-08T03:20:52.802662+0000","last_unstale":"2024-05-08T03:21:05.846757+0000","last_undegraded":"2024-05-08T03:21:05.846757+0000","last_fullsized":"2024-05-08T03:21:05.846757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:51.312283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":86524717,"num_objects":80,"num_object_clones":19,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":36,"num_read":6400,"num_read_kb":22013,"num_write":909,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"102'845","reported_seq":4103,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846942+0000","last_change":"2024-05-08T03:20:51.785227+0000","last_active":"2024-05-08T03:21:05.846942+0000","last_peered":"2024-05-08T03:21:05.846942+0000","last_clean":"2024-05-08T03:21:05.846942+0000","last_became_active":"2024-05-08T03:20:51.783022+0000","last_became_peered":"2024-05-08T03:20:51.783022+0000","last_unstale":"2024-05-08T03:21:05.846942+0000","last_undegraded":"2024-05-08T03:21:05.846942+0000","last_fullsized":"2024-05-08T03:21:05.846942+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:42:50.226983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":84249527,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":31,"num_read":7745,"num_read_kb":71002,"num_write":1188,"num_write_kb":48330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":48212116,"num_keys_recovered":59,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"102'375","reported_seq":3714,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847208+0000","last_change":"2024-05-08T03:20:40.780044+0000","last_active":"2024-05-08T03:21:05.847208+0000","last_peered":"2024-05-08T03:21:05.847208+0000","last_clean":"2024-05-08T03:21:05.847208+0000","last_became_active":"2024-05-08T03:20:40.779118+0000","last_became_peered":"2024-05-08T03:20:40.779118+0000","last_unstale":"2024-05-08T03:21:05.847208+0000","last_undegraded":"2024-05-08T03:21:05.847208+0000","last_fullsized":"2024-05-08T03:21:05.847208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:30.162952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":70256478,"num_objects":83,"num_object_clones":22,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":31,"num_read":3278,"num_read_kb":54569,"num_write":443,"num_write_kb":29246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":51071767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"102'517","reported_seq":3723,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.847086+0000","last_change":"2024-05-08T03:20:38.807093+0000","last_active":"2024-05-08T03:21:05.847086+0000","last_peered":"2024-05-08T03:21:05.847086+0000","last_clean":"2024-05-08T03:21:05.847086+0000","last_became_active":"2024-05-08T03:20:38.805719+0000","last_became_peered":"2024-05-08T03:20:38.805719+0000","last_unstale":"2024-05-08T03:21:05.847086+0000","last_undegraded":"2024-05-08T03:21:05.847086+0000","last_fullsized":"2024-05-08T03:21:05.847086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:29.670320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":27563758,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":3937,"num_read_kb":14749,"num_write":709,"num_write_kb":13908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":10326037,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"102'350","reported_seq":3463,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846246+0000","last_change":"2024-05-08T03:20:51.783432+0000","last_active":"2024-05-08T03:21:05.846246+0000","last_peered":"2024-05-08T03:21:05.846246+0000","last_clean":"2024-05-08T03:21:05.846246+0000","last_became_active":"2024-05-08T03:20:51.782569+0000","last_became_peered":"2024-05-08T03:20:51.782569+0000","last_unstale":"2024-05-08T03:21:05.846246+0000","last_undegraded":"2024-05-08T03:21:05.846246+0000","last_fullsized":"2024-05-08T03:21:05.846246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:43.399296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":70819277,"num_objects":73,"num_object_clones":16,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":32,"num_read":3884,"num_read_kb":35445,"num_write":357,"num_write_kb":18281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"102'407","reported_seq":5573,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846773+0000","last_change":"2024-05-08T03:20:38.806539+0000","last_active":"2024-05-08T03:21:05.846773+0000","last_peered":"2024-05-08T03:21:05.846773+0000","last_clean":"2024-05-08T03:21:05.846773+0000","last_became_active":"2024-05-08T03:20:38.805730+0000","last_became_peered":"2024-05-08T03:20:38.805730+0000","last_unstale":"2024-05-08T03:21:05.846773+0000","last_undegraded":"2024-05-08T03:21:05.846773+0000","last_fullsized":"2024-05-08T03:21:05.846773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:50.563335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":66655824,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":30,"num_read":6704,"num_read_kb":63814,"num_write":412,"num_write_kb":54964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":60512583,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"102'291","reported_seq":1838,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846624+0000","last_change":"2024-05-08T03:20:47.825128+0000","last_active":"2024-05-08T03:21:05.846624+0000","last_peered":"2024-05-08T03:21:05.846624+0000","last_clean":"2024-05-08T03:21:05.846624+0000","last_became_active":"2024-05-08T03:20:47.824316+0000","last_became_peered":"2024-05-08T03:20:47.824316+0000","last_unstale":"2024-05-08T03:21:05.846624+0000","last_undegraded":"2024-05-08T03:21:05.846624+0000","last_fullsized":"2024-05-08T03:21:05.846624+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:04:34.168774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":57536744,"num_objects":65,"num_object_clones":15,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":3292,"num_read_kb":26832,"num_write":285,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"102'339","reported_seq":3984,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-05-08T03:21:05.846975+0000","last_change":"2024-05-08T03:20:38.806543+0000","last_active":"2024-05-08T03:21:05.846975+0000","last_peered":"2024-05-08T03:21:05.846975+0000","last_clean":"2024-05-08T03:21:05.846975+0000","last_became_active":"2024-05-08T03:20:38.805711+0000","last_became_peered":"2024-05-08T03:20:38.805711+0000","last_unstale":"2024-05-08T03:21:05.846975+0000","last_undegraded":"2024-05-08T03:21:05.846975+0000","last_fullsized":"2024-05-08T03:21:05.846975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:32:01.986000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":32031328,"num_objects":76,"num_object_clones":21,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":37,"num_read":3799,"num_read_kb":38254,"num_write":434,"num_write_kb":23177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":40436240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1637461541,"num_objects":2200,"num_object_clones":486,"num_object_copies":4400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2200,"num_whiteouts":934,"num_read":161061,"num_read_kb":1062622,"num_write":18682,"num_write_kb":650594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4136,"num_bytes_recovered":610881052,"num_keys_recovered":712,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1196371968,"data_stored":1703877269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14938,"ondisk_log_size":14938,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542170,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":259888,"kb_used_data":184288,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94111952,"statfs":{"total":96636764160,"available":96370638848,"internally_reserved":0,"allocated":188710912,"data_stored":273100882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":0,"seq":455266533378,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388052,"kb_used_data":338816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":93983788,"statfs":{"total":96636764160,"available":96239398912,"internally_reserved":0,"allocated":346947584,"data_stored":478836932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542171,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212832,"kb_used_data":163604,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94159008,"statfs":{"total":96636764160,"available":96418824192,"internally_reserved":0,"allocated":167530496,"data_stored":243430470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315516,"kb_used_data":252396,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94056324,"statfs":{"total":96636764160,"available":96313675776,"internally_reserved":0,"allocated":258453504,"data_stored":360658701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542171,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":256589824,"data_stored":359463175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166248448,"data_stored":242819412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":135417856,"data_stored":199525057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":345665536,"data_stored":478225874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":186847232,"data_stored":271905356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:21:07.555 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T03:21:07.555 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:07.555 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T03:21:07.555 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.109034 2024-05-08T03:21:07.556 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.1090338353089827 2024-05-08T03:21:07.561 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.561 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.569 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:07.611 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:07.609+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:07.706 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:07.704+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:07.716 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.716 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:07.744 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:07.809 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:07.807+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:07.866 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:21:07.872 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:07.872 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:07.879 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:07.879 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:07.881 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:07.897 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:07.897 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:07.915 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:07.915 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:07.915 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:07.913+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:07.932 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:07.932 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:07.949 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:07.950 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:07.967 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:07.967 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:07.985 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:07.985 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:08.002 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:08.003 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:08.005 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:08.005 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:08.008 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:08.007+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:08.015 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:08.109 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:08.108+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:08.159 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:08.159 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:08.159 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:08.159 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:08.159 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:08.167 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:08.211 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:08.209+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:08.274 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 100.072343274, 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.275 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.279 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 100.072333824, 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:08.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:08.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:08.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.283 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 100.072321088, 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 100.072310512, 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.288 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:08.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:08.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 100.072301125, 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:08.292 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:08.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 100.072291945, 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:08.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:08.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:08.300 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841466909999994, 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841434672999995, 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:08.304 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:08.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:08.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841421342999993, 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.308 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:08.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841406368999998, 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.312 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:08.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841395059999996, 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.316 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:08.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841377390999995, 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:08.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:08.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.323 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841362192999995, 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.324 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.841348099000001, 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.328 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:08.332 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 92.066394126000006, 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:08.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:08.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:08.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.143500324000001, 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:08.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.143486835999994, 2024-05-08T03:21:08.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:08.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.143457854999994, 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.345 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.347 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.143431249000002, 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.349 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:08.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:08.353 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 90.710682988000002, 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:08.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.359 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:08.360 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:08.361 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:08.358+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:08.461 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:08.460+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:08.505 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:08.506 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:08.506 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:08.506 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:08.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:08.599 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:08.597+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:08.659 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:08.659 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:08.659 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:08.659 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:08.659 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:08.664 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:08.662+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:08.668 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:08.764 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:08.763+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:08.814 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.879867110000006, 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.815 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:08.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.821 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.019690579999999, 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.822 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.019664554999999, 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:08.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.830 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.831 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.01962254, 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:08.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.836 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.018639864000001, 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.837 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.018603947999999, 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.842 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:08.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:08.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.018578905999998, 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:08.847 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:08.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.016168986, 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:08.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.016144692000001, 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.862 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.016071575000002, 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:08.867 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.016043342, 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:08.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:08.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:08.872 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.016012594999999, 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:08.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:08.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:08.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.015975235999999, 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.877 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014373504999998, 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.882 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014319484000001, 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.887 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:08.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:08.891 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014286777999999, 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.892 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.896 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014262385999999, 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.897 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.901 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014242965000001, 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:08.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:08.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:08.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014231685999999, 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.906 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 18.014211992, 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:08.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:08.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:08.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:08.916 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:08.918 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:08.918 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.6 primary-affinity to 0.109034 (871452) 2024-05-08T03:21:08.919 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:08.916+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:09.018 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:09.017+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:09.063 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:09.064 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:09.064 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:09.064 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:09.073 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:09.156 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:09.155+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:09.218 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:09.218 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:09.218 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:09.218 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:09.218 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:09.220 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:09.219+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:09.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:09.321 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:09.320+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.273763787999997, 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:09.376 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:09.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.272043318000001, 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.381 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.385 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.272009877000002, 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:09.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.270584137999997, 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.391 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.270491524000001, 2024-05-08T03:21:09.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:09.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.269980850000003, 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.269953342000001, 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.404 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:09.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:09.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.269546910999999, 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.408 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:09.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:09.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.269091533000001, 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.416 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.268563286000003, 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.268090788000002, 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.268012656000003, 2024-05-08T03:21:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267937234000001, 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:09.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.433 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267860071000001, 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:09.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:09.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267782548, 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.438 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:09.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:09.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:09.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267755635999997, 2024-05-08T03:21:09.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:09.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.446 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267734425999997, 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:09.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:09.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.450 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267714646000002, 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:09.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:09.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.26716012, 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.455 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:09.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 61.267141107999997, 2024-05-08T03:21:09.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:09.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.463 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:09.465 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:09.465 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:09.463+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:09.565 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:09.564+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:09.611 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:09.612 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:09.612 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:09.612 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:09.621 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:09.666 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:09.665+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:09.766 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:09.767 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:09.767 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:09.767 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:09.767 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:09.767 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:09.765+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:09.775 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:09.868 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:09.866+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:09.883 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 70.885032611, 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:09.884 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:21:09.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 70.884965147000003, 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.893 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:21:09.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:09.896 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 70.825130814999994, 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:21:09.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.902 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.758622802000005, 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:21:09.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:21:09.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:09.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.732085155999997, 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.909 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:21:09.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:21:09.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:21:09.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.704173388000001, 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:21:09.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:21:09.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:21:09.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:21:09.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:21:09.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:09.985 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.625593195, 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.989 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620965818999998, 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.990 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620945184, 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.994 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:09.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:09.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:21:09.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620922933000003, 2024-05-08T03:21:09.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:09.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:21:10.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:21:10.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620902502, 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:21:10.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:10.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:21:10.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620882276000003, 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:21:10.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620861482000002, 2024-05-08T03:21:10.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:21:10.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:21:10.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:21:10.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.620835468000003, 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.440878548000001, 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.020 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.024 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.440859443999997, 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:10.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.440297870000002, 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.029 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.435084009000001, 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.033 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:21:10.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:21:10.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 41.434916668, 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:21:10.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:10.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:21:10.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 33.072163070999999, 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:10.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:21:10.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:21:10.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:10.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:10.049 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:10.049 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:21:10.050 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:10.051 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:10.049+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:10.052 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:10.152 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:10.150+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:10.169 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:10.169 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:10.169 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:10.170 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:10.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:10.217 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:10.217 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:10.234 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:10.235 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:10.251 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:10.251 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:10.263 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:10.261+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:10.268 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:10.268 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:10.285 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:10.285 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:10.302 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:10.303 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:10.321 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:10.321 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:10.327 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:10.327 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:10.336 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:10.337 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:10.338 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:10.353 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:10.352+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:10.454 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:10.452+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:10.482 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:10.482 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:10.491 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:10.555 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:10.553+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:10.636 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:10.636 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:10.645 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:10.689 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:10.687+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:10.750 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:10.750 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:10.756 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:10.755+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:10.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:10.857 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:10.856+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:10.874 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:21:10.906 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:10.906 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:10.915 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:10.959 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:10.957+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:11.021 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.021 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:11.060 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:11.058+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:11.160 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:11.159+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:11.174 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.175 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.183 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:11.261 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:11.260+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:11.290 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.290 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:11.362 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:11.361+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:11.405 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.405 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:11.498 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:11.496+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:11.559 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.559 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.564 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:11.563+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:11.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:11.666 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:11.664+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:11.712 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.712 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.721 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:11.766 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:11.764+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:11.835 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:11.836 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:11.845 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:11.887 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:11.885+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:11.908 INFO:teuthology.orchestra.run.smithi177.stdout:5436 write 0x3e0b308 thru 0x3e14100 (0x8df9 bytes) 2024-05-08T03:21:11.913 INFO:teuthology.orchestra.run.smithi177.stdout:5437 read 0x36ece09 thru 0x36f341d (0x6615 bytes) 2024-05-08T03:21:11.913 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:11.913 INFO:teuthology.orchestra.run.smithi177.stdout:5442 read 0x9f75d60 thru 0x9f84bca (0xee6b bytes) 2024-05-08T03:21:11.913 INFO:teuthology.orchestra.run.smithi177.stdout:5443 write 0x316b13f thru 0x316dcea (0x2bac bytes) 2024-05-08T03:21:11.918 INFO:teuthology.orchestra.run.smithi177.stdout:5444 read 0xbc7ff4 thru 0xbd2749 (0xa756 bytes) 2024-05-08T03:21:11.918 INFO:teuthology.orchestra.run.smithi177.stdout:5445 read 0x6350803 thru 0x635fbc2 (0xf3c0 bytes) 2024-05-08T03:21:11.918 INFO:teuthology.orchestra.run.smithi177.stdout:5446 punch from 0x16f73c2 to 0x16fc5dc, (0x521a bytes) 2024-05-08T03:21:11.919 INFO:teuthology.orchestra.run.smithi177.stdout:5447 trunc from 0xc55b074 to 0xc833327 2024-05-08T03:21:11.924 INFO:teuthology.orchestra.run.smithi177.stdout:5449 read 0x24a0d19 thru 0x24ac909 (0xbbf1 bytes) 2024-05-08T03:21:11.925 INFO:teuthology.orchestra.run.smithi177.stdout:5450 punch from 0x439c25b to 0x439e43b, (0x21e0 bytes) 2024-05-08T03:21:11.925 INFO:teuthology.orchestra.run.smithi177.stdout:5452 write 0x3c7251b thru 0x3c725ba (0xa0 bytes) 2024-05-08T03:21:11.943 INFO:teuthology.orchestra.run.smithi177.stdout:5453 trunc from 0xc833327 to 0x716d06 2024-05-08T03:21:11.967 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:11.965+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:11.974 INFO:teuthology.orchestra.run.smithi177.stdout:5454 punch from 0x436223 to 0x4385cd, (0x23aa bytes) 2024-05-08T03:21:11.974 INFO:teuthology.orchestra.run.smithi177.stdout:5455 read 0x1a4155 thru 0x1b2a1a (0xe8c6 bytes) 2024-05-08T03:21:11.995 INFO:teuthology.orchestra.run.smithi177.stdout:5456 writesame 0xcdad0e9 thru 0xcdb1826 data_size 0xb(0x473e bytes) 2024-05-08T03:21:11.995 INFO:teuthology.orchestra.run.smithi177.stdout:5457 punch from 0x80e4799 to 0x80ee895, (0xa0fc bytes) 2024-05-08T03:21:11.995 INFO:teuthology.orchestra.run.smithi177.stdout:5458 write 0xc71375 thru 0xc74cb5 (0x3941 bytes) 2024-05-08T03:21:12.007 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:12.008 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:12.008 INFO:teuthology.orchestra.run.smithi177.stdout:5459 trunc from 0xcdb1827 to 0x3033cd2 2024-05-08T03:21:12.014 INFO:teuthology.orchestra.run.smithi177.stdout:5460 read 0x17a4257 thru 0x17abee4 (0x7c8e bytes) 2024-05-08T03:21:12.018 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:12.022 INFO:teuthology.orchestra.run.smithi177.stdout:5461 writesame 0x4dab227 thru 0x4db14ba data_size 0x12(0x6294 bytes) 2024-05-08T03:21:12.022 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:12.023 INFO:teuthology.orchestra.run.smithi177.stdout:5463 trunc from 0x4db14bb to 0x5b8719d 2024-05-08T03:21:12.040 INFO:teuthology.orchestra.run.smithi177.stdout:5465 write 0xb39298f thru 0xb39983b (0x6ead bytes) 2024-05-08T03:21:12.048 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:12.048 INFO:teuthology.orchestra.run.smithi177.stdout:5467 read 0x155faaa thru 0x1567e68 (0x83bf bytes) 2024-05-08T03:21:12.048 INFO:teuthology.orchestra.run.smithi177.stdout:5468 punch from 0x6484039 to 0x6489e2f, (0x5df6 bytes) 2024-05-08T03:21:12.058 INFO:teuthology.orchestra.run.smithi177.stdout:5469 trunc from 0xb39983c to 0x2b6593d 2024-05-08T03:21:12.067 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:12.065+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:12.083 INFO:teuthology.orchestra.run.smithi177.stdout:5470 writesame 0x8c239f2 thru 0x8c2b9d5 data_size 0xa(0x7fe4 bytes) 2024-05-08T03:21:12.083 INFO:teuthology.orchestra.run.smithi177.stdout:5471 read 0xa6d99b thru 0xa778e1 (0x9f47 bytes) 2024-05-08T03:21:12.084 INFO:teuthology.orchestra.run.smithi177.stdout:5472 punch from 0x3ee5ec1 to 0x3eef104, (0x9243 bytes) 2024-05-08T03:21:12.084 INFO:teuthology.orchestra.run.smithi177.stdout:5473 write 0x441807 thru 0x44f67e (0xde78 bytes) 2024-05-08T03:21:12.100 INFO:teuthology.orchestra.run.smithi177.stdout:5474 write 0xe800e3d thru 0xe80e78e (0xd952 bytes) 2024-05-08T03:21:12.117 INFO:teuthology.orchestra.run.smithi177.stdout:5475 punch from 0xbcadc89 to 0xbcb5bbe, (0x7f35 bytes) 2024-05-08T03:21:12.118 INFO:teuthology.orchestra.run.smithi177.stdout:5476 punch from 0x839750a to 0x839a35f, (0x2e55 bytes) 2024-05-08T03:21:12.118 INFO:teuthology.orchestra.run.smithi177.stdout:5477 write 0x3d30498 thru 0x3d38193 (0x7cfc bytes) 2024-05-08T03:21:12.133 INFO:teuthology.orchestra.run.smithi177.stdout:5479 write 0x9b18ef1 thru 0x9b1bc1f (0x2d2f bytes) 2024-05-08T03:21:12.142 INFO:teuthology.orchestra.run.smithi177.stdout:5481 read 0xcfab0dc thru 0xcfb1d5a (0x6c7f bytes) 2024-05-08T03:21:12.142 INFO:teuthology.orchestra.run.smithi177.stdout:5485 read 0x4b01326 thru 0x4b08495 (0x7170 bytes) 2024-05-08T03:21:12.142 INFO:teuthology.orchestra.run.smithi177.stdout:5486 read 0x57b3608 thru 0x57b6e95 (0x388e bytes) 2024-05-08T03:21:12.143 INFO:teuthology.orchestra.run.smithi177.stdout:5487 read 0x6d3c707 thru 0x6d47d8a (0xb684 bytes) 2024-05-08T03:21:12.143 INFO:teuthology.orchestra.run.smithi177.stdout:5488 read 0x228502a thru 0x2287282 (0x2259 bytes) 2024-05-08T03:21:12.143 INFO:teuthology.orchestra.run.smithi177.stdout:5490 write 0x329d022 thru 0x329da86 (0xa65 bytes) 2024-05-08T03:21:12.143 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:12.144 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:12.155 INFO:teuthology.orchestra.run.smithi177.stdout:5491 trunc from 0xe80e78f to 0x7140022 2024-05-08T03:21:12.157 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:12.168 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:12.166+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:12.171 INFO:teuthology.orchestra.run.smithi177.stdout:5492 read 0x5489315 thru 0x548e9b1 (0x569d bytes) 2024-05-08T03:21:12.178 INFO:teuthology.orchestra.run.smithi177.stdout:5493 trunc from 0x7140022 to 0xc7bd406 2024-05-08T03:21:12.184 INFO:teuthology.orchestra.run.smithi177.stdout:5494 read 0x53fd73 thru 0x54517f (0x540d bytes) 2024-05-08T03:21:12.184 INFO:teuthology.orchestra.run.smithi177.stdout:5497 read 0xab93fd thru 0xac3343 (0x9f47 bytes) 2024-05-08T03:21:12.185 INFO:teuthology.orchestra.run.smithi177.stdout:5498 write 0x2e43819 thru 0x2e4cf9b (0x9783 bytes) 2024-05-08T03:21:12.200 INFO:teuthology.orchestra.run.smithi177.stdout:5499 writesame 0x121d335 thru 0x12262e6 data_size 0x3(0x8fb2 bytes) 2024-05-08T03:21:12.213 INFO:teuthology.orchestra.run.smithi177.stdout:5500 trunc from 0xc7bd406 to 0xaba0826 2024-05-08T03:21:12.221 INFO:teuthology.orchestra.run.smithi177.stdout:5501 write 0x5cbde4c thru 0x5cc1d02 (0x3eb7 bytes) 2024-05-08T03:21:12.237 INFO:teuthology.orchestra.run.smithi177.stdout:5502 read 0x57702a4 thru 0x577c148 (0xbea5 bytes) 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:5503 writesame 0xc77383c thru 0xc782b36 data_size 0x1(0xf2fb bytes) 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:5506 read 0xab1b197 thru 0xab2aeb0 (0xfd1a bytes) 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:5508 punch from 0x1e4f689 to 0x1e5d31e, (0xdc95 bytes) 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:5509 read 0x7971945 thru 0x79778ac (0x5f68 bytes) 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:5510 punch from 0x80e4602 to 0x80e5b45, (0x1543 bytes) 2024-05-08T03:21:12.246 INFO:teuthology.orchestra.run.smithi177.stdout:5512 trunc from 0xc782b37 to 0xd28738c 2024-05-08T03:21:12.253 INFO:teuthology.orchestra.run.smithi177.stdout:5513 punch from 0x74fd990 to 0x750ac30, (0xd2a0 bytes) 2024-05-08T03:21:12.253 INFO:teuthology.orchestra.run.smithi177.stdout:5514 write 0x783f840 thru 0x784f5ba (0xfd7b bytes) 2024-05-08T03:21:12.269 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:12.267+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:12.277 INFO:teuthology.orchestra.run.smithi177.stdout:5515 trunc from 0xd28738c to 0x2ffcb24 2024-05-08T03:21:12.294 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:12.294 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:12.304 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:12.318 INFO:teuthology.orchestra.run.smithi177.stdout:5516 write 0x7868b65 thru 0x787203d (0x94d9 bytes) 2024-05-08T03:21:12.343 INFO:teuthology.orchestra.run.smithi177.stdout:5518 write 0xd6ebcd2 thru 0xd6f5ba9 (0x9ed8 bytes) 2024-05-08T03:21:12.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:12.351 INFO:teuthology.orchestra.run.smithi177.stdout:5519 clone 42 order 23 su 131072 sc 3 2024-05-08T03:21:12.394 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:12.393+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:12.431 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:12.431 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:12.442 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:12.474 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:12.475 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:12.475 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:12.473+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:12.492 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:12.492 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:12.509 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:12.509 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:12.526 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:12.526 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:12.558 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:12.558 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:12.572 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:12.571+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:12.591 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:12.591 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:12.618 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 82.683603700999996, 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.823427170999999, 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.823401145999998, 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.823359131, 2024-05-08T03:21:12.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:12.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.822376455000001, 2024-05-08T03:21:12.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:12.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:12.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.822340538999999, 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.644 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:12.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.822315496999998, 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.649 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:12.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.819905577, 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:12.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:12.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:12.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.819881283000001, 2024-05-08T03:21:12.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:12.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:12.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:12.662 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.819808166000001, 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:12.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:12.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.819779933, 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:12.667 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.819749185999999, 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.819711826999999, 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:12.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:12.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.681 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.818110096000002, 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.682 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:12.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.818056075000001, 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.687 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:12.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:12.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:12.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.818023368999999, 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.692 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.696 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.817998976999998, 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.697 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:12.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:12.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:12.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.701 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.817979556000001, 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:12.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:12.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:12.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.817968276999999, 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.706 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:12.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 21.817948583, 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:12.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:12.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:12.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:12.715 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:12.716 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:12.717 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:12.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:12.718 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:12.719 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:12.719 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:12.715+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:12.818 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:12.816+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:12.865 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:12.865 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:12.865 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:12.865 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:12.874 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:12.889 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:21:12.919 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:12.917+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:12.991 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:12.992 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:12.992 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:12.992 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:12.992 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:13.002 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:13.046 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:13.045+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:13.102 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-05-08T03:21:13.120 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:13.118+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.019883536999998, 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.123 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:13.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:13.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.127 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.018163067000003, 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.018129626000004, 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.132 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:13.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:13.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.016703887000006, 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:13.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:13.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.016611272999995, 2024-05-08T03:21:13.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:13.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:13.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:13.145 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.016100598999998, 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.146 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.150 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.016073090999996, 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:13.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.015666659999994, 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.155 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:13.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:13.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.015211281999996, 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:13.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:13.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.014683035000004, 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:13.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.168 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.014210536999997, 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.014132404999998, 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.173 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:13.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:13.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:13.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.014056983000003, 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.178 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.182 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013979820000003, 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:13.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:13.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:13.186 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013902297000001, 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.187 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013875385000006, 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013854175000006, 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013834395000003, 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013279869000002, 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 65.013260857000006, 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.214 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:13.214 INFO:teuthology.orchestra.run.smithi177.stdout:5521 write 0x937ec8 thru 0x939b1a (0x1c53 bytes) 2024-05-08T03:21:13.215 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:13.224 INFO:teuthology.orchestra.run.smithi177.stdout:5522 trunc from 0xd6f5baa to 0x4c661eb 2024-05-08T03:21:13.224 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:13.223+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:13.261 INFO:teuthology.orchestra.run.smithi177.stdout:5523 writesame 0xd703c61 thru 0xd713c38 data_size 0xc(0xffd8 bytes) 2024-05-08T03:21:13.262 INFO:teuthology.orchestra.run.smithi177.stdout:5525 read 0x57627e2 thru 0x576db3a (0xb359 bytes) 2024-05-08T03:21:13.262 INFO:teuthology.orchestra.run.smithi177.stdout:5528 write 0x97d0a81 thru 0x97d3970 (0x2ef0 bytes) 2024-05-08T03:21:13.265 INFO:teuthology.orchestra.run.smithi177.stdout:5529 punch from 0xf940d3 to 0xf9c822, (0x874f bytes) 2024-05-08T03:21:13.265 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:13.265 INFO:teuthology.orchestra.run.smithi177.stdout:5532 writesame 0xc2d3a0d thru 0xc2dccdb data_size 0x3b(0x92cf bytes) 2024-05-08T03:21:13.270 INFO:teuthology.orchestra.run.smithi177.stdout:5533 read 0x7e63e41 thru 0x7e6a833 (0x69f3 bytes) 2024-05-08T03:21:13.270 INFO:teuthology.orchestra.run.smithi177.stdout:5534 write 0xc021aff thru 0xc02e29a (0xc79c bytes) 2024-05-08T03:21:13.271 INFO:teuthology.orchestra.run.smithi177.stdout:5535 clone 43 order 23 su 65536 sc 9 2024-05-08T03:21:13.321 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:13.320+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:13.334 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:13.334 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:13.334 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:13.334 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:13.343 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:13.423 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:13.421+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:13.488 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:13.488 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:13.488 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:13.488 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:13.488 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:13.496 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:13.540 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:13.539+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:13.603 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 74.604613982000004, 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.604 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:21:13.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:21:13.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:21:13.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:21:13.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 74.604546518000006, 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.610 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:21:13.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:21:13.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:21:13.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:13.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.617 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 74.544712185999998, 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.618 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:21:13.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.478204172999995, 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:21:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:21:13.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:21:13.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:21:13.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.451666527, 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.630 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:21:13.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:21:13.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:21:13.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:13.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.423754759000005, 2024-05-08T03:21:13.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:13.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.639 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.345174565999997, 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:13.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340547190000002, 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:13.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:21:13.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.647 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340526554999997, 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:13.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.651 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340504304, 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.652 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:21:13.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340483872999997, 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:13.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:21:13.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340463647, 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.661 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.665 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340442852999999, 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:21:13.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:13.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:21:13.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:21:13.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.340416839, 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:21:13.671 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:21:13.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:21:13.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.160459918999997, 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.675 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:21:13.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:21:13.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.160440815000001, 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.679 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:13.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.159879240999999, 2024-05-08T03:21:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.687 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.154665379999997, 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:21:13.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:13.693 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 45.154498039000003, 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:13.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.791744442000002, 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:13.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:21:13.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:13.703 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:13.705 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:13.706 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:13.703+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:13.806 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:13.805+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:13.812 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:13.812 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:13.812 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:13.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:13.888 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:21:13.906 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:13.905+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:13.920 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:13.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T03:21:13.929 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:13.929 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:13.929 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:13.929 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:13.929 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:13.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:13.939 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone42 from 0xd713c39 (overlap 0x4c661eb) to 0x7a3cfd 2024-05-08T03:21:14.022 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:14.020+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:14.086 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:14.086 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:14.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:14.086 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:14.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.086 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.310744877000001, 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:14.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.296121672000002, 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:14.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:14.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:14.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.295312393, 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:14.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:14.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:14.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.294373115999999, 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.104 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:14.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:14.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.292740045999999, 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:14.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.116 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.29130155, 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:14.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.121 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.291281661999999, 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.122 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:14.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.291257742999999, 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.127 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:14.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.131 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.290286562999999, 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:14.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:14.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:14.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:14.136 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.290274954000001, 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:14.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:14.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:14.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.290265236, 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.141 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.290254774000001, 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:14.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:14.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:14.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.287677828, 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:14.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.287647581000002, 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:14.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.160 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.287620564000001, 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:14.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.165 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.287601571, 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:14.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:14.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.287571512, 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.170 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.285787048, 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.175 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:14.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.179 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.285775224999998, 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.180 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:14.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:14.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 23.285763368000001, 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:14.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:14.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.189 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:14.191 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:14.191 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:14.189+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:14.292 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:14.290+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:14.367 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:14.367 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:14.367 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:14.368 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:14.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:14.422 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:14.420+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:14.458 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-05-08T03:21:14.484 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:14.485 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:14.485 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:14.485 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:14.485 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:14.493 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:14.494 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:14.491+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:14.594 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:14.592+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 98.429688623999994, 2024-05-08T03:21:14.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:14.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:14.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:14.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.645 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12895 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2490368~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.510008107000004, 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015686835999999999, 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12895 2024-05-08T03:21:14.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128100+0000", 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7700000000000005e-07 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128102+0000", 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1289999999999999e-06 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128111+0000", 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0900000000000005e-06 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128116+0000", 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2449999999999998e-06 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141178+0000", 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.013061899 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.141233+0000", 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5276999999999999e-05 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143787+0000", 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025537189999999999 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12896 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 4063232~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509984036000006, 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016328386, 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.650 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12896 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128124+0000", 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128126+0000", 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128135+0000", 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0069999999999997e-06 2024-05-08T03:21:14.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128141+0000", 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-06 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143832+0000", 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.015690975999999999 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.143916+0000", 2024-05-08T03:21:14.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3496000000000002e-05 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144452+0000", 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053639400000000004 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509958312999998, 2024-05-08T03:21:14.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:14.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509926596, 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:14.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509901491999997, 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:14.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509876687000002, 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:14.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:14.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.670 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509848142999999, 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.671 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.674 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509825621000005, 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:14.678 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509795280999995, 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.679 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:14.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:14.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 97.509774100000001, 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.683 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 83.697132163999996, 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.687 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:14.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.692 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.508457442999998, 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.693 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:14.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:14.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.486254099999996, 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.484557981999998, 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.703 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.707 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.464853194, 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:14.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:14.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:14.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:14.713 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.464829234000007, 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:14.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:14.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:14.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:14.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.424237688999995, 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:14.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:14.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:14.723 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.411463792000006, 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.727 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 55.150724857999997, 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:14.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:14.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:14.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:14.732 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:14.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:14.733 INFO:teuthology.orchestra.run.smithi177.stdout:5536 writesame 0x235b25a thru 0x235e380 data_size 0x1(0x3127 bytes) 2024-05-08T03:21:14.734 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:14.735 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:14.731+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:14.748 INFO:teuthology.orchestra.run.smithi177.stdout:5537 read 0x9e4798b thru 0x9e4e693 (0x6d09 bytes) 2024-05-08T03:21:14.759 INFO:teuthology.orchestra.run.smithi177.stdout:5539 trunc from 0xd713c39 to 0x44b42fa 2024-05-08T03:21:14.836 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:14.834+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:14.897 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T03:21:14.908 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:14.908 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:14.908 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:14.908 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:14.911 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:14.911 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:14.912 INFO:teuthology.orchestra.run.smithi177.stdout:5540 write 0x1a89575 thru 0x1a9186e (0x82fa bytes) 2024-05-08T03:21:14.913 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T03:21:14.918 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:14.933 INFO:teuthology.orchestra.run.smithi177.stdout:5543 write 0xc9e22e5 thru 0xc9ee2b2 (0xbfce bytes) 2024-05-08T03:21:14.935 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:14.933+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:14.939 INFO:teuthology.orchestra.run.smithi177.stdout:5544 trunc from 0xc9ee2b3 to 0xa2174a8 2024-05-08T03:21:14.939 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:14.940 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:14.946 INFO:teuthology.orchestra.run.smithi177.stdout:5545 writesame 0x68282c3 thru 0x682bf45 data_size 0x1(0x3c83 bytes) 2024-05-08T03:21:14.957 INFO:teuthology.orchestra.run.smithi177.stdout:5548 write 0x421344b thru 0x4217234 (0x3dea bytes) 2024-05-08T03:21:14.959 INFO:teuthology.orchestra.run.smithi177.stdout:5550 writesame 0x3fc4fb thru 0x3ff5d7 data_size 0x1(0x30dd bytes) 2024-05-08T03:21:14.960 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:14.960 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:14.966 INFO:teuthology.orchestra.run.smithi177.stdout:5551 writesame 0x64eb299 thru 0x64eb535 data_size 0x1(0x29d bytes) 2024-05-08T03:21:14.978 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:14.978 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:14.980 INFO:teuthology.orchestra.run.smithi177.stdout:5552 write 0xd562d35 thru 0xd56aefa (0x81c6 bytes) 2024-05-08T03:21:14.995 INFO:teuthology.orchestra.run.smithi177.stdout:5553 trunc from 0xd56aefb to 0x45acb50 2024-05-08T03:21:14.996 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:14.996 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:15.010 INFO:teuthology.orchestra.run.smithi177.stdout:5554 read 0x1c853ac thru 0x1c8b09b (0x5cf0 bytes) 2024-05-08T03:21:15.010 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.010 INFO:teuthology.orchestra.run.smithi177.stdout:5556 punch from 0xe16785 to 0xe25294, (0xeb0f bytes) 2024-05-08T03:21:15.011 INFO:teuthology.orchestra.run.smithi177.stdout:5557 punch from 0x1acc485 to 0x1ad05de, (0x4159 bytes) 2024-05-08T03:21:15.017 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:15.017 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:15.020 INFO:teuthology.orchestra.run.smithi177.stdout:5558 write 0x72f6aa7 thru 0x730234c (0xb8a6 bytes) 2024-05-08T03:21:15.028 INFO:teuthology.orchestra.run.smithi177.stdout:5559 trunc from 0x730234d to 0x9398be9 2024-05-08T03:21:15.031 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:15.031 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:15.031 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:15.031 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:15.031 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:15.034 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.034 INFO:teuthology.orchestra.run.smithi177.stdout:5562 read 0x28d616c thru 0x28e4411 (0xe2a6 bytes) 2024-05-08T03:21:15.035 INFO:teuthology.orchestra.run.smithi177.stdout:5563 flatten 2024-05-08T03:21:15.035 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:15.033+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:15.037 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:15.037 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:15.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:15.042 INFO:teuthology.orchestra.run.smithi177.stdout:5564 read 0x1614a73 thru 0x161bfcf (0x755d bytes) 2024-05-08T03:21:15.044 INFO:teuthology.orchestra.run.smithi177.stdout:5565 trunc from 0x9398be9 to 0x8039825 2024-05-08T03:21:15.055 INFO:teuthology.orchestra.run.smithi177.stdout:5566 punch from 0x1afe381 to 0x1b06116, (0x7d95 bytes) 2024-05-08T03:21:15.055 INFO:teuthology.orchestra.run.smithi177.stdout:5567 write 0x1007292 thru 0x1007a58 (0x7c7 bytes) 2024-05-08T03:21:15.058 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:15.058 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:15.072 INFO:teuthology.orchestra.run.smithi177.stdout:5568 writesame 0xec885df thru 0xec927a2 data_size 0x44(0xa1c4 bytes) 2024-05-08T03:21:15.072 INFO:teuthology.orchestra.run.smithi177.stdout:5570 writesame 0x7233520 thru 0x7234f4d data_size 0x3(0x1a2e bytes) 2024-05-08T03:21:15.094 INFO:teuthology.orchestra.run.smithi177.stdout:5571 trunc from 0xec927a3 to 0x2cf9a4 2024-05-08T03:21:15.132 INFO:teuthology.orchestra.run.smithi177.stdout:5572 write 0x974149f thru 0x974c61d (0xb17f bytes) 2024-05-08T03:21:15.136 INFO:teuthology.orchestra.run.smithi177.stdout:5573 punch from 0x3dc2953 to 0x3dc8aef, (0x619c bytes) 2024-05-08T03:21:15.136 INFO:teuthology.orchestra.run.smithi177.stdout:5574 read 0x4da8174 thru 0x4db3b92 (0xba1f bytes) 2024-05-08T03:21:15.136 INFO:teuthology.orchestra.run.smithi177.stdout:5575 write 0x323f206 thru 0x3244f02 (0x5cfd bytes) 2024-05-08T03:21:15.136 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:15.134+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:15.149 INFO:teuthology.orchestra.run.smithi177.stdout:5576 write 0xdf5d722 thru 0xdf6b98d (0xe26c bytes) 2024-05-08T03:21:15.152 INFO:teuthology.orchestra.run.smithi177.stdout:5577 write 0x8227679 thru 0x82286a3 (0x102b bytes) 2024-05-08T03:21:15.153 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.153 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.156 INFO:teuthology.orchestra.run.smithi177.stdout:5578 write 0x7eb3f5e thru 0x7eb574b (0x17ee bytes) 2024-05-08T03:21:15.159 INFO:teuthology.orchestra.run.smithi177.stdout:5579 writesame 0x2ab1c8 thru 0x2b38cb data_size 0x2(0x8704 bytes) 2024-05-08T03:21:15.164 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:15.165 INFO:teuthology.orchestra.run.smithi177.stdout:5580 write 0x2841751 thru 0x2847806 (0x60b6 bytes) 2024-05-08T03:21:15.167 INFO:teuthology.orchestra.run.smithi177.stdout:5582 write 0xc199e4d thru 0xc19ba5a (0x1c0e bytes) 2024-05-08T03:21:15.170 INFO:teuthology.orchestra.run.smithi177.stdout:5583 write 0x4eafa5 thru 0x4f297d (0x79d9 bytes) 2024-05-08T03:21:15.172 INFO:teuthology.orchestra.run.smithi177.stdout:5584 punch from 0x99c582d to 0x99cd8b7, (0x808a bytes) 2024-05-08T03:21:15.172 INFO:teuthology.orchestra.run.smithi177.stdout:5585 write 0x7c6f158 thru 0x7c74702 (0x55ab bytes) 2024-05-08T03:21:15.175 INFO:teuthology.orchestra.run.smithi177.stdout:5586 read 0xcd58828 thru 0xcd64504 (0xbcdd bytes) 2024-05-08T03:21:15.184 INFO:teuthology.orchestra.run.smithi177.stdout:5587 trunc from 0xdf6b98e to 0x66c4cf6 2024-05-08T03:21:15.215 INFO:teuthology.orchestra.run.smithi177.stdout:5590 write 0xce887e8 thru 0xce8cde5 (0x45fe bytes) 2024-05-08T03:21:15.221 INFO:teuthology.orchestra.run.smithi177.stdout:5591 punch from 0x13c15df to 0x13ccc92, (0xb6b3 bytes) 2024-05-08T03:21:15.230 INFO:teuthology.orchestra.run.smithi177.stdout:5593 trunc from 0xce8cde6 to 0x55fb715 2024-05-08T03:21:15.237 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:15.235+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:15.241 INFO:teuthology.orchestra.run.smithi177.stdout:5594 write 0x1cf7b5 thru 0x1d6a60 (0x72ac bytes) 2024-05-08T03:21:15.247 INFO:teuthology.orchestra.run.smithi177.stdout:5595 writesame 0x5a2ed34 thru 0x5a3a2a5 data_size 0x19(0xb572 bytes) 2024-05-08T03:21:15.249 INFO:teuthology.orchestra.run.smithi177.stdout:5596 read 0x41a395a thru 0x41ad1d7 (0x987e bytes) 2024-05-08T03:21:15.249 INFO:teuthology.orchestra.run.smithi177.stdout:5597 read 0x464b856 thru 0x464c4ff (0xcaa bytes) 2024-05-08T03:21:15.250 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.250 INFO:teuthology.orchestra.run.smithi177.stdout:5602 writesame 0xa1c189 thru 0xa1c244 data_size 0x2(0xbc bytes) 2024-05-08T03:21:15.261 INFO:teuthology.orchestra.run.smithi177.stdout:5603 write 0x9b99afb thru 0x9ba5fa2 (0xc4a8 bytes) 2024-05-08T03:21:15.266 INFO:teuthology.orchestra.run.smithi177.stdout:5604 writesame 0x8f319f1 thru 0x8f3b005 data_size 0x9(0x9615 bytes) 2024-05-08T03:21:15.272 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.274 INFO:teuthology.orchestra.run.smithi177.stdout:5606 trunc from 0x9ba5fa3 to 0x75f4aa2 2024-05-08T03:21:15.278 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.278 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.288 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:15.290 INFO:teuthology.orchestra.run.smithi177.stdout:5607 write 0x1e7e1cb thru 0x1e82246 (0x407c bytes) 2024-05-08T03:21:15.298 INFO:teuthology.orchestra.run.smithi177.stdout:5608 trunc from 0x75f4aa2 to 0x25c0af9 2024-05-08T03:21:15.335 INFO:teuthology.orchestra.run.smithi177.stdout:5610 write 0xc13418d thru 0xc13d0cf (0x8f43 bytes) 2024-05-08T03:21:15.337 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:15.336+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:15.338 INFO:teuthology.orchestra.run.smithi177.stdout:5611 punch from 0x3ce100c to 0x3ce68c8, (0x58bc bytes) 2024-05-08T03:21:15.338 INFO:teuthology.orchestra.run.smithi177.stdout:5612 writesame 0x762418f thru 0x7629f36 data_size 0x6f(0x5da8 bytes) 2024-05-08T03:21:15.341 INFO:teuthology.orchestra.run.smithi177.stdout:5613 read 0xae504e7 thru 0xae55606 (0x5120 bytes) 2024-05-08T03:21:15.341 INFO:teuthology.orchestra.run.smithi177.stdout:5614 write 0x23af109 thru 0x23b1edd (0x2dd5 bytes) 2024-05-08T03:21:15.352 INFO:teuthology.orchestra.run.smithi177.stdout:5615 trunc from 0xc13d0d0 to 0x6e325e5 2024-05-08T03:21:15.363 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.363 INFO:teuthology.orchestra.run.smithi177.stdout:5617 read 0x27d5394 thru 0x27e0847 (0xb4b4 bytes) 2024-05-08T03:21:15.364 INFO:teuthology.orchestra.run.smithi177.stdout:5618 trunc from 0x6e325e5 to 0x6c5e8c9 2024-05-08T03:21:15.370 INFO:teuthology.orchestra.run.smithi177.stdout:5620 read 0x463a45b thru 0x464959e (0xf144 bytes) 2024-05-08T03:21:15.370 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.370 INFO:teuthology.orchestra.run.smithi177.stdout:5622 write 0x51d5dfe thru 0x51e41c8 (0xe3cb bytes) 2024-05-08T03:21:15.384 INFO:teuthology.orchestra.run.smithi177.stdout:5623 trunc from 0x6c5e8c9 to 0x1756881 2024-05-08T03:21:15.399 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.399 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.407 INFO:teuthology.orchestra.run.smithi177.stdout:5624 trunc from 0x1756881 to 0x9989d2f 2024-05-08T03:21:15.409 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:15.417 INFO:teuthology.orchestra.run.smithi177.stdout:5625 trunc from 0x9989d2f to 0x5622605 2024-05-08T03:21:15.423 INFO:teuthology.orchestra.run.smithi177.stdout:5626 trunc from 0x5622605 to 0x5860b3f 2024-05-08T03:21:15.435 INFO:teuthology.orchestra.run.smithi177.stdout:5629 write 0x691508c thru 0x6918cd1 (0x3c46 bytes) 2024-05-08T03:21:15.441 INFO:teuthology.orchestra.run.smithi177.stdout:5631 read 0x3200478 thru 0x3210284 (0xfe0d bytes) 2024-05-08T03:21:15.441 INFO:teuthology.orchestra.run.smithi177.stdout:5634 read 0x3e7e55d thru 0x3e82796 (0x423a bytes) 2024-05-08T03:21:15.442 INFO:teuthology.orchestra.run.smithi177.stdout:5635 write 0x2f4f7f1 thru 0x2f5624e (0x6a5e bytes) 2024-05-08T03:21:15.446 INFO:teuthology.orchestra.run.smithi177.stdout:5638 read 0xb8753a thru 0xb941de (0xcca5 bytes) 2024-05-08T03:21:15.446 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.446 INFO:teuthology.orchestra.run.smithi177.stdout:5642 write 0x3402eb0 thru 0x3410048 (0xd199 bytes) 2024-05-08T03:21:15.452 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:15.451+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:15.455 INFO:teuthology.orchestra.run.smithi177.stdout:5643 trunc from 0x6918cd2 to 0xe22ec0 2024-05-08T03:21:15.476 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.476 INFO:teuthology.orchestra.run.smithi177.stdout:5646 read 0x527c07 thru 0x531582 (0x997c bytes) 2024-05-08T03:21:15.476 INFO:teuthology.orchestra.run.smithi177.stdout:5647 read 0x32e1c0 thru 0x331e72 (0x3cb3 bytes) 2024-05-08T03:21:15.476 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:15.477 INFO:teuthology.orchestra.run.smithi177.stdout:5650 read 0x29f68b thru 0x2a600f (0x6985 bytes) 2024-05-08T03:21:15.490 INFO:teuthology.orchestra.run.smithi177.stdout:5651 write 0x6eb3f19 thru 0x6ec00e2 (0xc1ca bytes) 2024-05-08T03:21:15.496 INFO:teuthology.orchestra.run.smithi177.stdout:5652 write 0x4e69d33 thru 0x4e77830 (0xdafe bytes) 2024-05-08T03:21:15.500 INFO:teuthology.orchestra.run.smithi177.stdout:5654 clone 44 order 19 su 131072 sc 10 2024-05-08T03:21:15.518 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.518 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.527 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:15.571 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:15.569+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:15.633 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.633 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.641 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:15.642 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:15.639+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:15.742 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:15.741+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:15.786 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.786 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.795 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:15.844 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:15.842+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:15.899 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T03:21:15.903 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:21:15.906 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:15.906 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:15.918 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-08T03:21:15.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:15.944 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:15.942+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:16.045 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:16.043+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:16.066 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-05-08T03:21:16.084 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.084 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.087 INFO:teuthology.orchestra.run.smithi177.stdout:5655 punch from 0x21b8c2f to 0x21c8ba9, (0xff7a bytes) 2024-05-08T03:21:16.098 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:16.102 INFO:teuthology.orchestra.run.smithi177.stdout:5656 write 0xa07a61b thru 0xa07eedc (0x48c2 bytes) 2024-05-08T03:21:16.109 INFO:teuthology.orchestra.run.smithi177.stdout:skipping zero size writesame 2024-05-08T03:21:16.109 INFO:teuthology.orchestra.run.smithi177.stdout:5660 write 0x1c9fc75 thru 0x1caec21 (0xefad bytes) 2024-05-08T03:21:16.123 INFO:teuthology.orchestra.run.smithi177.stdout:5662 trunc from 0xa07eedd to 0x72a4b56 2024-05-08T03:21:16.137 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.145 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:16.144+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:16.152 INFO:teuthology.orchestra.run.smithi177.stdout:5664 write 0xe4d5d8a thru 0xe4dc92f (0x6ba6 bytes) 2024-05-08T03:21:16.157 INFO:teuthology.orchestra.run.smithi177.stdout:5666 flatten 2024-05-08T03:21:16.247 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:16.245+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:16.252 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.253 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.263 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:16.265 INFO:teuthology.orchestra.run.smithi177.stdout:5667 write 0x7450bd4 thru 0x745783b (0x6c68 bytes) 2024-05-08T03:21:16.272 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.272 INFO:teuthology.orchestra.run.smithi177.stdout:5670 read 0xa4de94a thru 0xa4eb870 (0xcf27 bytes) 2024-05-08T03:21:16.272 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.272 INFO:teuthology.orchestra.run.smithi177.stdout:5672 write 0xac0ef62 thru 0xac14ed3 (0x5f72 bytes) 2024-05-08T03:21:16.278 INFO:teuthology.orchestra.run.smithi177.stdout:5673 write 0xcaccb6b thru 0xcad7bdb (0xb071 bytes) 2024-05-08T03:21:16.302 INFO:teuthology.orchestra.run.smithi177.stdout:5674 trunc from 0xe4dc930 to 0x33956de 2024-05-08T03:21:16.324 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.336 INFO:teuthology.orchestra.run.smithi177.stdout:5677 writesame 0x589a352 thru 0x589dcd7 data_size 0x25(0x3986 bytes) 2024-05-08T03:21:16.336 INFO:teuthology.orchestra.run.smithi177.stdout:5678 punch from 0x11d3910 to 0x11d72eb, (0x39db bytes) 2024-05-08T03:21:16.336 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.336 INFO:teuthology.orchestra.run.smithi177.stdout:5680 write 0x3918f2c thru 0x391da60 (0x4b35 bytes) 2024-05-08T03:21:16.341 INFO:teuthology.orchestra.run.smithi177.stdout:5681 punch from 0x44102d4 to 0x441d052, (0xcd7e bytes) 2024-05-08T03:21:16.341 INFO:teuthology.orchestra.run.smithi177.stdout:5683 read 0x3bec6d5 thru 0x3bf00a3 (0x39cf bytes) 2024-05-08T03:21:16.347 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:16.346+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:16.355 INFO:teuthology.orchestra.run.smithi177.stdout:5684 write 0xbbf69fb thru 0xbc004e9 (0x9aef bytes) 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:5685 punch from 0x64da762 to 0x64e2420, (0x7cbe bytes) 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:5691 punch from 0x9e3a95d to 0x9e4583a, (0xaedd bytes) 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:5692 read 0x6d38baa thru 0x6d399eb (0xe42 bytes) 2024-05-08T03:21:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:5693 write 0x615328f thru 0x615372e (0x4a0 bytes) 2024-05-08T03:21:16.369 INFO:teuthology.orchestra.run.smithi177.stdout:5694 write 0xc116a88 thru 0xc11c85d (0x5dd6 bytes) 2024-05-08T03:21:16.372 INFO:teuthology.orchestra.run.smithi177.stdout:5695 write 0x82efde3 thru 0x82f48d5 (0x4af3 bytes) 2024-05-08T03:21:16.376 INFO:teuthology.orchestra.run.smithi177.stdout:5696 punch from 0xbfae3ed to 0xbfaf819, (0x142c bytes) 2024-05-08T03:21:16.377 INFO:teuthology.orchestra.run.smithi177.stdout:5697 read 0xa577759 thru 0xa57899b (0x1243 bytes) 2024-05-08T03:21:16.377 INFO:teuthology.orchestra.run.smithi177.stdout:5698 punch from 0xdd560a to 0xddec81, (0x9677 bytes) 2024-05-08T03:21:16.383 INFO:teuthology.orchestra.run.smithi177.stdout:5699 write 0xc54beba thru 0xc5564c9 (0xa610 bytes) 2024-05-08T03:21:16.385 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.385 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.396 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:16.400 INFO:teuthology.orchestra.run.smithi177.stdout:5700 write 0xe0068ee thru 0xe015b3f (0xf252 bytes) 2024-05-08T03:21:16.407 INFO:teuthology.orchestra.run.smithi177.stdout:5701 read 0x671e8de thru 0x672a293 (0xb9b6 bytes) 2024-05-08T03:21:16.407 INFO:teuthology.orchestra.run.smithi177.stdout:5702 punch from 0x99a5897 to 0x99aefbc, (0x9725 bytes) 2024-05-08T03:21:16.407 INFO:teuthology.orchestra.run.smithi177.stdout:5704 punch from 0x5f50050 to 0x5f59fad, (0x9f5d bytes) 2024-05-08T03:21:16.407 INFO:teuthology.orchestra.run.smithi177.stdout:5705 punch from 0x7597df6 to 0x759f190, (0x739a bytes) 2024-05-08T03:21:16.408 INFO:teuthology.orchestra.run.smithi177.stdout:5706 read 0x57ab735 thru 0x57b4ed8 (0x97a4 bytes) 2024-05-08T03:21:16.408 INFO:teuthology.orchestra.run.smithi177.stdout:5708 writesame 0x235715e thru 0x235a0ca data_size 0x13(0x2f6d bytes) 2024-05-08T03:21:16.411 INFO:teuthology.orchestra.run.smithi177.stdout:5709 write 0x6475891 thru 0x6476ced (0x145d bytes) 2024-05-08T03:21:16.416 INFO:teuthology.orchestra.run.smithi177.stdout:5713 punch from 0xd1cb2ff to 0xd1d652e, (0xb22f bytes) 2024-05-08T03:21:16.416 INFO:teuthology.orchestra.run.smithi177.stdout:5714 punch from 0x94bbca6 to 0x94c1464, (0x57be bytes) 2024-05-08T03:21:16.416 INFO:teuthology.orchestra.run.smithi177.stdout:5715 writesame 0x8b4f7a1 thru 0x8b52eb0 data_size 0x8(0x3710 bytes) 2024-05-08T03:21:16.421 INFO:teuthology.orchestra.run.smithi177.stdout:5716 write 0xccca7cf thru 0xcccba62 (0x1294 bytes) 2024-05-08T03:21:16.429 INFO:teuthology.orchestra.run.smithi177.stdout:5717 writesame 0xb511d4f thru 0xb512af5 data_size 0x5(0xda7 bytes) 2024-05-08T03:21:16.435 INFO:teuthology.orchestra.run.smithi177.stdout:5718 write 0x3690c4d thru 0x3698b28 (0x7edc bytes) 2024-05-08T03:21:16.438 INFO:teuthology.orchestra.run.smithi177.stdout:5719 punch from 0xc5d92d6 to 0xc5dea86, (0x57b0 bytes) 2024-05-08T03:21:16.438 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.438 INFO:teuthology.orchestra.run.smithi177.stdout:5721 write 0x23cdeb5 thru 0x23d3f0f (0x605b bytes) 2024-05-08T03:21:16.444 INFO:teuthology.orchestra.run.smithi177.stdout:5723 read 0xb120d22 thru 0xb12910b (0x83ea bytes) 2024-05-08T03:21:16.444 INFO:teuthology.orchestra.run.smithi177.stdout:5724 writesame 0x753dbbc thru 0x7548969 data_size 0xb(0xadae bytes) 2024-05-08T03:21:16.448 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:16.446+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:16.454 INFO:teuthology.orchestra.run.smithi177.stdout:5725 write 0x66698cd thru 0x666e1fb (0x492f bytes) 2024-05-08T03:21:16.460 INFO:teuthology.orchestra.run.smithi177.stdout:5726 punch from 0xd64174 to 0xd6cbcc, (0x8a58 bytes) 2024-05-08T03:21:16.460 INFO:teuthology.orchestra.run.smithi177.stdout:5728 read 0x79f9f3b thru 0x7a08bb1 (0xec77 bytes) 2024-05-08T03:21:16.460 INFO:teuthology.orchestra.run.smithi177.stdout:5729 read 0xd447084 thru 0xd452acd (0xba4a bytes) 2024-05-08T03:21:16.460 INFO:teuthology.orchestra.run.smithi177.stdout:5730 read 0x48cd0c4 thru 0x48d7098 (0x9fd5 bytes) 2024-05-08T03:21:16.460 INFO:teuthology.orchestra.run.smithi177.stdout:5731 punch from 0x63989a0 to 0x639be99, (0x34f9 bytes) 2024-05-08T03:21:16.479 INFO:teuthology.orchestra.run.smithi177.stdout:5733 trunc from 0xe015b40 to 0x20be7d 2024-05-08T03:21:16.527 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.527 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.528 INFO:teuthology.orchestra.run.smithi177.stdout:5734 writesame 0xba4a144 thru 0xba52c57 data_size 0x12(0x8b14 bytes) 2024-05-08T03:21:16.529 INFO:teuthology.orchestra.run.smithi177.stdout:5735 trunc from 0xba52c58 to 0xca61b97 2024-05-08T03:21:16.535 INFO:teuthology.orchestra.run.smithi177.stdout:5736 writesame 0x5cf10fb thru 0x5cf1d01 data_size 0x1(0xc07 bytes) 2024-05-08T03:21:16.538 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:16.553 INFO:teuthology.orchestra.run.smithi177.stdout:5738 trunc from 0xca61b97 to 0x1aaff65 2024-05-08T03:21:16.553 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:16.551+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:16.565 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.565 INFO:teuthology.orchestra.run.smithi177.stdout:5742 read 0x124ea31 thru 0x1256743 (0x7d13 bytes) 2024-05-08T03:21:16.580 INFO:teuthology.orchestra.run.smithi177.stdout:5743 trunc from 0x1aaff65 to 0xe1baa99 2024-05-08T03:21:16.586 INFO:teuthology.orchestra.run.smithi177.stdout:5745 punch from 0x7245e13 to 0x724c346, (0x6533 bytes) 2024-05-08T03:21:16.586 INFO:teuthology.orchestra.run.smithi177.stdout:5746 trunc from 0xe1baa99 to 0xe81ee1a 2024-05-08T03:21:16.592 INFO:teuthology.orchestra.run.smithi177.stdout:5747 read 0x698f170 thru 0x69949bb (0x584c bytes) 2024-05-08T03:21:16.592 INFO:teuthology.orchestra.run.smithi177.stdout:5748 write 0x1273927 thru 0x1281a31 (0xe10b bytes) 2024-05-08T03:21:16.597 INFO:teuthology.orchestra.run.smithi177.stdout:5749 read 0x5e663d2 thru 0x5e698ff (0x352e bytes) 2024-05-08T03:21:16.597 INFO:teuthology.orchestra.run.smithi177.stdout:5750 read 0x67e534e thru 0x67e960e (0x42c1 bytes) 2024-05-08T03:21:16.597 INFO:teuthology.orchestra.run.smithi177.stdout:5751 writesame 0xc3303d thru 0xc3dcb7 data_size 0x1(0xac7b bytes) 2024-05-08T03:21:16.628 INFO:teuthology.orchestra.run.smithi177.stdout:5752 writesame 0xa5e1667 thru 0xa5f0ea6 data_size 0x40(0xf840 bytes) 2024-05-08T03:21:16.632 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.633 INFO:teuthology.orchestra.run.smithi177.stdout:5756 read 0xfdd88 thru 0x108c94 (0xaf0d bytes) 2024-05-08T03:21:16.635 INFO:teuthology.orchestra.run.smithi177.stdout:5757 trunc from 0xe81ee1a to 0xca71ca9 2024-05-08T03:21:16.642 INFO:teuthology.orchestra.run.smithi177.stdout:5758 trunc from 0xca71ca9 to 0xcc4f849 2024-05-08T03:21:16.646 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.646 INFO:teuthology.orchestra.run.smithi177.stdout:5763 write 0xb0ba97f thru 0xb0c0152 (0x57d4 bytes) 2024-05-08T03:21:16.650 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:16.649+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:16.651 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.651 INFO:teuthology.orchestra.run.smithi177.stdout:5766 writesame 0xbd91538 thru 0xbd93790 data_size 0x3(0x2259 bytes) 2024-05-08T03:21:16.655 INFO:teuthology.orchestra.run.smithi177.stdout:5767 write 0x3a9b61b thru 0x3aa35f8 (0x7fde bytes) 2024-05-08T03:21:16.660 INFO:teuthology.orchestra.run.smithi177.stdout:5768 read 0x6c1ae64 thru 0x6c27790 (0xc92d bytes) 2024-05-08T03:21:16.660 INFO:teuthology.orchestra.run.smithi177.stdout:5770 write 0x9ce1ba6 thru 0x9ceb1f5 (0x9650 bytes) 2024-05-08T03:21:16.664 INFO:teuthology.orchestra.run.smithi177.stdout:5771 writesame 0x7c9476f thru 0x7c9df24 data_size 0x3(0x97b6 bytes) 2024-05-08T03:21:16.670 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.670 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.673 INFO:teuthology.orchestra.run.smithi177.stdout:5772 write 0xb7b68fb thru 0xb7c5ff5 (0xf6fb bytes) 2024-05-08T03:21:16.679 INFO:teuthology.orchestra.run.smithi177.stdout:5773 writesame 0x31d081e thru 0x31d762c data_size 0x73(0x6e0f bytes) 2024-05-08T03:21:16.680 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:16.682 INFO:teuthology.orchestra.run.smithi177.stdout:5774 read 0xc985add thru 0xc995452 (0xf976 bytes) 2024-05-08T03:21:16.682 INFO:teuthology.orchestra.run.smithi177.stdout:5775 punch from 0x4469ff6 to 0x4478a1d, (0xea27 bytes) 2024-05-08T03:21:16.682 INFO:teuthology.orchestra.run.smithi177.stdout:5776 write 0x5b8c45f thru 0x5b9a1e5 (0xdd87 bytes) 2024-05-08T03:21:16.685 INFO:teuthology.orchestra.run.smithi177.stdout:5777 read 0x9d643a2 thru 0x9d71b8a (0xd7e9 bytes) 2024-05-08T03:21:16.685 INFO:teuthology.orchestra.run.smithi177.stdout:5778 writesame 0x649de19 thru 0x64a910e data_size 0x1(0xb2f6 bytes) 2024-05-08T03:21:16.705 INFO:teuthology.orchestra.run.smithi177.stdout:5779 trunc from 0xcc4f849 to 0xd9e507b 2024-05-08T03:21:16.711 INFO:teuthology.orchestra.run.smithi177.stdout:5781 write 0xc65119d thru 0xc65cbcd (0xba31 bytes) 2024-05-08T03:21:16.715 INFO:teuthology.orchestra.run.smithi177.stdout:5782 read 0x989cb6d thru 0x98a4fca (0x845e bytes) 2024-05-08T03:21:16.715 INFO:teuthology.orchestra.run.smithi177.stdout:5786 writesame 0x62ed8b5 thru 0x62f84a5 data_size 0x1(0xabf1 bytes) 2024-05-08T03:21:16.743 INFO:teuthology.orchestra.run.smithi177.stdout:5787 writesame 0x66b267f thru 0x66bc2d9 data_size 0x1(0x9c5b bytes) 2024-05-08T03:21:16.751 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:16.749+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:16.766 INFO:teuthology.orchestra.run.smithi177.stdout:5789 read 0xbf6816a thru 0xbf6ad4c (0x2be3 bytes) 2024-05-08T03:21:16.766 INFO:teuthology.orchestra.run.smithi177.stdout:5790 punch from 0x3fc5bfa to 0x3fd55ea, (0xf9f0 bytes) 2024-05-08T03:21:16.766 INFO:teuthology.orchestra.run.smithi177.stdout:5791 read 0x1aa7509 thru 0x1ab5ecf (0xe9c7 bytes) 2024-05-08T03:21:16.766 INFO:teuthology.orchestra.run.smithi177.stdout:5792 writesame 0x673f04f thru 0x6744523 data_size 0x39(0x54d5 bytes) 2024-05-08T03:21:16.769 INFO:teuthology.orchestra.run.smithi177.stdout:5793 writesame 0x245a931 thru 0x245e4e8 data_size 0x38(0x3bb8 bytes) 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:5795 read 0x2edcda0 thru 0x2ee4af9 (0x7d5a bytes) 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:5797 read 0x3890218 thru 0x389e087 (0xde70 bytes) 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:5798 punch from 0xa2dd8 to 0xaa065, (0x728d bytes) 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:5801 read 0x21777dd thru 0x217ddc6 (0x65ea bytes) 2024-05-08T03:21:16.773 INFO:teuthology.orchestra.run.smithi177.stdout:5802 writesame 0x983e2a6 thru 0x983f9be data_size 0x1b(0x1719 bytes) 2024-05-08T03:21:16.776 INFO:teuthology.orchestra.run.smithi177.stdout:5803 write 0xa1f6b29 thru 0xa1f73e0 (0x8b8 bytes) 2024-05-08T03:21:16.783 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.783 INFO:teuthology.orchestra.run.smithi177.stdout:5807 punch from 0x6067c70 to 0x6073c49, (0xbfd9 bytes) 2024-05-08T03:21:16.783 INFO:teuthology.orchestra.run.smithi177.stdout:5808 read 0x2fc7745 thru 0x2fce7e8 (0x70a4 bytes) 2024-05-08T03:21:16.783 INFO:teuthology.orchestra.run.smithi177.stdout:5809 punch from 0x193c624 to 0x1947a0d, (0xb3e9 bytes) 2024-05-08T03:21:16.783 INFO:teuthology.orchestra.run.smithi177.stdout:5810 write 0x2b51278 thru 0x2b55b42 (0x48cb bytes) 2024-05-08T03:21:16.787 INFO:teuthology.orchestra.run.smithi177.stdout:5812 read 0xc87932a thru 0xc87b1d6 (0x1ead bytes) 2024-05-08T03:21:16.787 INFO:teuthology.orchestra.run.smithi177.stdout:5813 write 0x61419af thru 0x6143d93 (0x23e5 bytes) 2024-05-08T03:21:16.791 INFO:teuthology.orchestra.run.smithi177.stdout:5814 write 0x43d327d thru 0x43d8573 (0x52f7 bytes) 2024-05-08T03:21:16.802 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.802 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.805 INFO:teuthology.orchestra.run.smithi177.stdout:5815 trunc from 0xd9e507b to 0x5fdb3c8 2024-05-08T03:21:16.814 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:16.831 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.831 INFO:teuthology.orchestra.run.smithi177.stdout:5817 read 0x14d676b thru 0x14e3758 (0xcfee bytes) 2024-05-08T03:21:16.831 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.845 INFO:teuthology.orchestra.run.smithi177.stdout:5819 writesame 0xdb2bf83 thru 0xdb36a72 data_size 0x14(0xaaf0 bytes) 2024-05-08T03:21:16.852 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:16.851+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:16.855 INFO:teuthology.orchestra.run.smithi177.stdout:5820 trunc from 0xdb36a73 to 0x5f6edce 2024-05-08T03:21:16.861 INFO:teuthology.orchestra.run.smithi177.stdout:5821 punch from 0x1d1f138 to 0x1d1f80e, (0x6d6 bytes) 2024-05-08T03:21:16.871 INFO:teuthology.orchestra.run.smithi177.stdout:5822 trunc from 0x5f6edce to 0xcd58951 2024-05-08T03:21:16.876 INFO:teuthology.orchestra.run.smithi177.stdout:5823 read 0x7135adc thru 0x7143fab (0xe4d0 bytes) 2024-05-08T03:21:16.876 INFO:teuthology.orchestra.run.smithi177.stdout:5825 write 0xc9cce11 thru 0xc9d5582 (0x8772 bytes) 2024-05-08T03:21:16.880 INFO:teuthology.orchestra.run.smithi177.stdout:5827 read 0xbd92247 thru 0xbd9e709 (0xc4c3 bytes) 2024-05-08T03:21:16.880 INFO:teuthology.orchestra.run.smithi177.stdout:5828 punch from 0x855a4a to 0x864550, (0xeb06 bytes) 2024-05-08T03:21:16.880 INFO:teuthology.orchestra.run.smithi177.stdout:5830 write 0xb6df0b1 thru 0xb6e5dc5 (0x6d15 bytes) 2024-05-08T03:21:16.888 INFO:teuthology.orchestra.run.smithi177.stdout:5832 read 0xcaefce1 thru 0xcaf1fb8 (0x22d8 bytes) 2024-05-08T03:21:16.889 INFO:teuthology.orchestra.run.smithi177.stdout:5833 punch from 0x8c2d215 to 0x8c3a985, (0xd770 bytes) 2024-05-08T03:21:16.889 INFO:teuthology.orchestra.run.smithi177.stdout:5834 punch from 0x20a7c4e to 0x20a7dcb, (0x17d bytes) 2024-05-08T03:21:16.889 INFO:teuthology.orchestra.run.smithi177.stdout:5836 writesame 0x7da9be3 thru 0x7daca6d data_size 0x1(0x2e8b bytes) 2024-05-08T03:21:16.897 INFO:teuthology.orchestra.run.smithi177.stdout:5837 writesame 0x35e64a8 thru 0x35f5527 data_size 0x34(0xf080 bytes) 2024-05-08T03:21:16.904 INFO:teuthology.orchestra.run.smithi177.stdout:5838 read 0xbb3904a thru 0xbb3cc1d (0x3bd4 bytes) 2024-05-08T03:21:16.904 INFO:teuthology.orchestra.run.smithi177.stdout:5839 punch from 0xb9867bb to 0xb98d586, (0x6dcb bytes) 2024-05-08T03:21:16.904 INFO:teuthology.orchestra.run.smithi177.stdout:5840 read 0x4bf1902 thru 0x4bf971a (0x7e19 bytes) 2024-05-08T03:21:16.904 INFO:teuthology.orchestra.run.smithi177.stdout:5841 write 0x2e87584 thru 0x2e8e03f (0x6abc bytes) 2024-05-08T03:21:16.904 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.6 to 1 (10000) 2024-05-08T03:21:16.904 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:21:16.914 INFO:teuthology.orchestra.run.smithi177.stdout:5842 read 0x5aefd7 thru 0x5be3ac (0xf3d6 bytes) 2024-05-08T03:21:16.914 INFO:teuthology.orchestra.run.smithi177.stdout:5843 read 0xc17a72c thru 0xc17b06a (0x93f bytes) 2024-05-08T03:21:16.914 INFO:teuthology.orchestra.run.smithi177.stdout:5844 write 0x9890e5e thru 0x9895bc1 (0x4d64 bytes) 2024-05-08T03:21:16.917 INFO:teuthology.orchestra.run.smithi177.stdout:5845 write 0x97e4322 thru 0x97f0abe (0xc79d bytes) 2024-05-08T03:21:16.922 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T03:21:16.923 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.923 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.923 INFO:teuthology.orchestra.run.smithi177.stdout:5848 read 0x249f2ab thru 0x24a973e (0xa494 bytes) 2024-05-08T03:21:16.923 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.923 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.923 INFO:teuthology.orchestra.run.smithi177.stdout:5851 read 0xca5c18f thru 0xca5c65e (0x4d0 bytes) 2024-05-08T03:21:16.927 INFO:teuthology.orchestra.run.smithi177.stdout:5852 trunc from 0xcd58951 to 0x8e18937 2024-05-08T03:21:16.939 INFO:teuthology.orchestra.run.smithi177.stdout:5853 write 0x6a8d640 thru 0x6a98162 (0xab23 bytes) 2024-05-08T03:21:16.942 INFO:teuthology.orchestra.run.smithi177.stdout:5855 punch from 0x6805ac9 to 0x6806590, (0xac7 bytes) 2024-05-08T03:21:16.942 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:16.946 INFO:teuthology.orchestra.run.smithi177.stdout:5857 trunc from 0x8e18937 to 0x612682b 2024-05-08T03:21:16.953 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:16.951+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:16.964 INFO:teuthology.orchestra.run.smithi177.stdout:5858 write 0x726c743 thru 0x726eb11 (0x23cf bytes) 2024-05-08T03:21:16.965 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:16.965 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:16.968 INFO:teuthology.orchestra.run.smithi177.stdout:5861 write 0x1dd548e thru 0x1ddddfb (0x896e bytes) 2024-05-08T03:21:16.978 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:16.982 INFO:teuthology.orchestra.run.smithi177.stdout:5862 write 0xab310bd thru 0xab35885 (0x47c9 bytes) 2024-05-08T03:21:16.985 INFO:teuthology.orchestra.run.smithi177.stdout:5863 clone 45 order 24 su 524288 sc 13 2024-05-08T03:21:17.054 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:17.052+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:17.076 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:17.093 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:17.093 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:17.103 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:17.187 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:17.186+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:17.220 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:17.220 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:17.250 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:17.250 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:17.255 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:17.254+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:17.267 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:17.267 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:17.275 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:17.275 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:17.284 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:17.284 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:17.285 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:17.301 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:17.301 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:17.318 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:17.318 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:17.335 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:17.336 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:17.353 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:17.354 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:17.357 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:17.355+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:17.392 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:17.392 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:17.392 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:17.392 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:17.400 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:17.457 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:17.456+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:17.546 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:17.547 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:17.547 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:17.547 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:17.547 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:17.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:17.558 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:17.557+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:17.658 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:17.656+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:17.661 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.886265125000001, 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.662 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:17.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:17.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.871641919999998, 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.668 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.670 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.671 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.672 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:17.673 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.674 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.870832641, 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.675 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.676 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.677 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.678 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.679 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.680 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.869893363999999, 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.681 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.682 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:17.683 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.684 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.685 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.686 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.868260293999999, 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.687 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.688 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.689 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.866821798, 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.693 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:17.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:17.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:17.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.697 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.86680191, 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:17.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.702 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.866777990999999, 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:17.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:17.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.865806810999999, 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.707 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:17.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:17.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.711 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.865795202000001, 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:17.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:17.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:17.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.865785484, 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.716 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:17.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.720 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.865775022000001, 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:17.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:17.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.863198076, 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.725 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:17.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.863167828999998, 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.730 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:17.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:17.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:17.734 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.863140812000001, 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:17.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:17.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:17.739 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.863121819, 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:17.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:17.743 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.86309176, 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.744 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.861307296, 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.748 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:17.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:17.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:17.752 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.861295472999998, 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.753 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:17.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:17.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 26.861283616000001, 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.758 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:17.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:17.762 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:17.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:17.764 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:17.762+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:17.865 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:17.863+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:17.910 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:17.910 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:17.910 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:17.910 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:17.912 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T03:21:17.920 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:17.926 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T03:21:17.926 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:21:18.007 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:18.005+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:18.066 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:18.066 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:18.066 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:18.066 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:18.066 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:18.067 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:18.065+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:18.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:18.086 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-05-08T03:21:18.170 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:18.168+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:18.210 INFO:teuthology.orchestra.run.smithi177.stdout:5865 read 0x163d186 thru 0x16437b4 (0x662f bytes) 2024-05-08T03:21:18.210 INFO:teuthology.orchestra.run.smithi177.stdout:5866 write 0x952e074 thru 0x9535faf (0x7f3c bytes) 2024-05-08T03:21:18.222 INFO:teuthology.orchestra.run.smithi177.stdout:5867 trunc from 0xab35886 to 0x78a5c6c 2024-05-08T03:21:18.223 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:18.223 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:18.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:18.223 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:18.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.223 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 102.01259231, 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.224 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.09286199899999, 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.229 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:18.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:18.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:18.233 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.09283028199999, 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:18.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.09280517800001, 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:18.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.241 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.092780373, 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.242 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.243 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.244 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.245 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.09275182899999, 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.246 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.247 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.248 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.249 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.092729307, 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.250 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.251 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.252 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.253 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.092698967, 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:18.254 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.255 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.256 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.092677786, 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.258 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:18.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:18.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:18.261 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 87.280035850000004, 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.262 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.264 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.265 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.266 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:18.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.091361129000006, 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.268 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:18.269 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:18.270 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.272 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.069157786000005, 2024-05-08T03:21:18.273 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.274 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.275 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.276 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.067461668000007, 2024-05-08T03:21:18.277 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.278 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:18.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:18.280 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:18.281 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.047756879999994, 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.282 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.283 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:18.284 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:18.285 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:18.286 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.047732920000001, 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.287 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.288 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.289 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:18.290 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.291 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 78.007141375000003, 2024-05-08T03:21:18.292 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.293 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.294 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.295 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:18.296 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.994367478000001, 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:18.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:18.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.300 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 58.733628543999998, 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.301 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.302 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.303 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.375324145, 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.305 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:21:18.306 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:21:18.307 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:21:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 3.3737054080000002, 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.309 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.310 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.311 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.312 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.313 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:18.315 INFO:teuthology.orchestra.run.smithi177.stdout:5868 punch from 0x5086c1e to 0x5094e55, (0xe237 bytes) 2024-05-08T03:21:18.315 INFO:teuthology.orchestra.run.smithi177.stdout:5869 write 0xa2fabc7 thru 0xa306964 (0xbd9e bytes) 2024-05-08T03:21:18.315 INFO:teuthology.orchestra.run.smithi177.stdout:5870 punch from 0x1d01575 to 0x1d019e7, (0x472 bytes) 2024-05-08T03:21:18.315 INFO:teuthology.orchestra.run.smithi177.stdout:5871 read 0x285c332 thru 0x286949c (0xd16b bytes) 2024-05-08T03:21:18.315 INFO:teuthology.orchestra.run.smithi177.stdout:5872 trunc from 0xa306965 to 0x1a3fd85 2024-05-08T03:21:18.316 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:18.316 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:18.313+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:18.317 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:18.317 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:21:18.319 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-05-08T03:21:18.193453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1688464117,"num_objects":2260,"num_object_clones":495,"num_object_copies":4520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2260,"num_whiteouts":945,"num_read":173223,"num_read_kb":1135294,"num_write":20121,"num_write_kb":719973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":612652156,"num_keys_recovered":712,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15722,"ondisk_log_size":15722,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1236844,"kb_used_data":967180,"kb_used_omap":6,"kb_used_meta":241273,"kb_avail":376250516,"statfs":{"total":386547056640,"available":385280528384,"internally_reserved":0,"allocated":990392320,"data_stored":1400024010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6379,"internal_metadata":247064341},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50412208,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":11,"num_read":12046,"num_read_kb":72572,"num_write":1280,"num_write_kb":66963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000670"},"pg_stats":[{"pgid":"3.1f","version":"112'312","reported_seq":2990,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.902833+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:21:15.902833+0000","last_peered":"2024-05-08T03:21:15.902833+0000","last_clean":"2024-05-08T03:21:15.902833+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:21:15.902833+0000","last_undegraded":"2024-05-08T03:21:15.902833+0000","last_fullsized":"2024-05-08T03:21:15.902833+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:16.621119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":57941618,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":25,"num_read":2874,"num_read_kb":31303,"num_write":365,"num_write_kb":16834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"115'268","reported_seq":2409,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.207417+0000","last_change":"2024-05-08T03:20:50.785288+0000","last_active":"2024-05-08T03:21:16.207417+0000","last_peered":"2024-05-08T03:21:16.207417+0000","last_clean":"2024-05-08T03:21:16.207417+0000","last_became_active":"2024-05-08T03:20:50.783719+0000","last_became_peered":"2024-05-08T03:20:50.783719+0000","last_unstale":"2024-05-08T03:21:16.207417+0000","last_undegraded":"2024-05-08T03:21:16.207417+0000","last_fullsized":"2024-05-08T03:21:16.207417+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:32:30.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":34604698,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":2277,"num_read_kb":7335,"num_write":312,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"116'441","reported_seq":6722,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.044871+0000","last_change":"2024-05-08T03:20:39.774078+0000","last_active":"2024-05-08T03:21:16.044871+0000","last_peered":"2024-05-08T03:21:16.044871+0000","last_clean":"2024-05-08T03:21:16.044871+0000","last_became_active":"2024-05-08T03:20:39.773251+0000","last_became_peered":"2024-05-08T03:20:39.773251+0000","last_unstale":"2024-05-08T03:21:16.044871+0000","last_undegraded":"2024-05-08T03:21:16.044871+0000","last_fullsized":"2024-05-08T03:21:16.044871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:17:20.312838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":72489935,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":20,"num_read":6929,"num_read_kb":44818,"num_write":525,"num_write_kb":14062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"116'879","reported_seq":3781,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.040592+0000","last_change":"2024-05-08T03:20:40.781096+0000","last_active":"2024-05-08T03:21:16.040592+0000","last_peered":"2024-05-08T03:21:16.040592+0000","last_clean":"2024-05-08T03:21:16.040592+0000","last_became_active":"2024-05-08T03:20:40.779478+0000","last_became_peered":"2024-05-08T03:20:40.779478+0000","last_unstale":"2024-05-08T03:21:16.040592+0000","last_undegraded":"2024-05-08T03:21:16.040592+0000","last_fullsized":"2024-05-08T03:21:16.040592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:36:32.730237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":58611786,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":25,"num_read":5504,"num_read_kb":51895,"num_write":1263,"num_write_kb":51140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"116'394","reported_seq":3301,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.108710+0000","last_change":"2024-05-08T03:20:40.780143+0000","last_active":"2024-05-08T03:21:16.108710+0000","last_peered":"2024-05-08T03:21:16.108710+0000","last_clean":"2024-05-08T03:21:16.108710+0000","last_became_active":"2024-05-08T03:20:40.779325+0000","last_became_peered":"2024-05-08T03:20:40.779325+0000","last_unstale":"2024-05-08T03:21:16.108710+0000","last_undegraded":"2024-05-08T03:21:16.108710+0000","last_fullsized":"2024-05-08T03:21:16.108710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:44.350271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":42359262,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":29,"num_read":4180,"num_read_kb":21566,"num_write":479,"num_write_kb":13537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7457235,"num_keys_recovered":60,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"115'533","reported_seq":5180,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.200587+0000","last_change":"2024-05-08T03:20:39.775128+0000","last_active":"2024-05-08T03:21:16.200587+0000","last_peered":"2024-05-08T03:21:16.200587+0000","last_clean":"2024-05-08T03:21:16.200587+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:21:16.200587+0000","last_undegraded":"2024-05-08T03:21:16.200587+0000","last_fullsized":"2024-05-08T03:21:16.200587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:49.821278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":57834027,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":5450,"num_read_kb":44144,"num_write":737,"num_write_kb":50757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"116'950","reported_seq":3789,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.179503+0000","last_change":"2024-05-08T03:20:51.782224+0000","last_active":"2024-05-08T03:21:16.179503+0000","last_peered":"2024-05-08T03:21:16.179503+0000","last_clean":"2024-05-08T03:21:16.179503+0000","last_became_active":"2024-05-08T03:20:51.780774+0000","last_became_peered":"2024-05-08T03:20:51.780774+0000","last_unstale":"2024-05-08T03:21:16.179503+0000","last_undegraded":"2024-05-08T03:21:16.179503+0000","last_fullsized":"2024-05-08T03:21:16.179503+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:20.051241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":51288125,"num_objects":70,"num_object_clones":11,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":6174,"num_read_kb":22121,"num_write":1534,"num_write_kb":11725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"116'451","reported_seq":8478,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.078690+0000","last_change":"2024-05-08T03:20:50.779322+0000","last_active":"2024-05-08T03:21:16.078690+0000","last_peered":"2024-05-08T03:21:16.078690+0000","last_clean":"2024-05-08T03:21:16.078690+0000","last_became_active":"2024-05-08T03:20:50.777845+0000","last_became_peered":"2024-05-08T03:20:50.777845+0000","last_unstale":"2024-05-08T03:21:16.078690+0000","last_undegraded":"2024-05-08T03:21:16.078690+0000","last_fullsized":"2024-05-08T03:21:16.078690+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:05:06.140301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":77310646,"num_objects":77,"num_object_clones":14,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":33,"num_read":9764,"num_read_kb":57949,"num_write":458,"num_write_kb":15068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":18070131,"num_keys_recovered":60,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"115'329","reported_seq":4276,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903895+0000","last_change":"2024-05-08T03:20:50.781739+0000","last_active":"2024-05-08T03:21:15.903895+0000","last_peered":"2024-05-08T03:21:15.903895+0000","last_clean":"2024-05-08T03:21:15.903895+0000","last_became_active":"2024-05-08T03:20:50.780815+0000","last_became_peered":"2024-05-08T03:20:50.780815+0000","last_unstale":"2024-05-08T03:21:15.903895+0000","last_undegraded":"2024-05-08T03:21:15.903895+0000","last_fullsized":"2024-05-08T03:21:15.903895+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:46.121738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":28956805,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":4659,"num_read_kb":23165,"num_write":347,"num_write_kb":4530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"115'221","reported_seq":3266,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903719+0000","last_change":"2024-05-08T03:20:51.783981+0000","last_active":"2024-05-08T03:21:15.903719+0000","last_peered":"2024-05-08T03:21:15.903719+0000","last_clean":"2024-05-08T03:21:15.903719+0000","last_became_active":"2024-05-08T03:20:51.783155+0000","last_became_peered":"2024-05-08T03:20:51.783155+0000","last_unstale":"2024-05-08T03:21:15.903719+0000","last_undegraded":"2024-05-08T03:21:15.903719+0000","last_fullsized":"2024-05-08T03:21:15.903719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:11:21.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":92523192,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":29,"num_read":2596,"num_read_kb":61854,"num_write":213,"num_write_kb":45811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":54103866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"115'292","reported_seq":7014,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900549+0000","last_change":"2024-05-08T03:20:50.785464+0000","last_active":"2024-05-08T03:21:15.900549+0000","last_peered":"2024-05-08T03:21:15.900549+0000","last_clean":"2024-05-08T03:21:15.900549+0000","last_became_active":"2024-05-08T03:20:50.783826+0000","last_became_peered":"2024-05-08T03:20:50.783826+0000","last_unstale":"2024-05-08T03:21:15.900549+0000","last_undegraded":"2024-05-08T03:21:15.900549+0000","last_fullsized":"2024-05-08T03:21:15.900549+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:28.162671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":52883839,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":41,"num_read":6383,"num_read_kb":36940,"num_write":300,"num_write_kb":7491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6817760,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"116'689","reported_seq":4486,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.151059+0000","last_change":"2024-05-08T03:20:37.789834+0000","last_active":"2024-05-08T03:21:16.151059+0000","last_peered":"2024-05-08T03:21:16.151059+0000","last_clean":"2024-05-08T03:21:16.151059+0000","last_became_active":"2024-05-08T03:20:37.788932+0000","last_became_peered":"2024-05-08T03:20:37.788932+0000","last_unstale":"2024-05-08T03:21:16.151059+0000","last_undegraded":"2024-05-08T03:21:16.151059+0000","last_fullsized":"2024-05-08T03:21:16.151059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:34.769709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":31294263,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":5975,"num_read_kb":21317,"num_write":906,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":2507487,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":52,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.902441+0000","last_change":"2024-05-08T03:20:47.820453+0000","last_active":"2024-05-08T03:21:15.902441+0000","last_peered":"2024-05-08T03:21:15.902441+0000","last_clean":"2024-05-08T03:21:15.902441+0000","last_became_active":"2024-05-08T03:20:47.820180+0000","last_became_peered":"2024-05-08T03:20:47.820180+0000","last_unstale":"2024-05-08T03:21:15.902441+0000","last_undegraded":"2024-05-08T03:21:15.902441+0000","last_fullsized":"2024-05-08T03:21:15.902441+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:33.427981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"116'304","reported_seq":3493,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.117882+0000","last_change":"2024-05-08T03:20:50.782204+0000","last_active":"2024-05-08T03:21:16.117882+0000","last_peered":"2024-05-08T03:21:16.117882+0000","last_clean":"2024-05-08T03:21:16.117882+0000","last_became_active":"2024-05-08T03:20:50.781137+0000","last_became_peered":"2024-05-08T03:20:50.781137+0000","last_unstale":"2024-05-08T03:21:16.117882+0000","last_undegraded":"2024-05-08T03:21:16.117882+0000","last_fullsized":"2024-05-08T03:21:16.117882+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:10:10.778552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":17582148,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":32,"num_read":2984,"num_read_kb":11310,"num_write":331,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":52,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900614+0000","last_change":"2024-05-08T03:20:47.821931+0000","last_active":"2024-05-08T03:21:15.900614+0000","last_peered":"2024-05-08T03:21:15.900614+0000","last_clean":"2024-05-08T03:21:15.900614+0000","last_became_active":"2024-05-08T03:20:47.821642+0000","last_became_peered":"2024-05-08T03:20:47.821642+0000","last_unstale":"2024-05-08T03:21:15.900614+0000","last_undegraded":"2024-05-08T03:21:15.900614+0000","last_fullsized":"2024-05-08T03:21:15.900614+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:42.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"116'690","reported_seq":4544,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.156222+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:21:16.156222+0000","last_peered":"2024-05-08T03:21:16.156222+0000","last_clean":"2024-05-08T03:21:16.156222+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:21:16.156222+0000","last_undegraded":"2024-05-08T03:21:16.156222+0000","last_fullsized":"2024-05-08T03:21:16.156222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.761270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":27356756,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":30,"num_read":6706,"num_read_kb":33422,"num_write":896,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900639+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:21:15.900639+0000","last_peered":"2024-05-08T03:21:15.900639+0000","last_clean":"2024-05-08T03:21:15.900639+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:21:15.900639+0000","last_undegraded":"2024-05-08T03:21:15.900639+0000","last_fullsized":"2024-05-08T03:21:15.900639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"116'256","reported_seq":5190,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.201450+0000","last_change":"2024-05-08T03:20:50.784522+0000","last_active":"2024-05-08T03:21:16.201450+0000","last_peered":"2024-05-08T03:21:16.201450+0000","last_clean":"2024-05-08T03:21:16.201450+0000","last_became_active":"2024-05-08T03:20:50.783297+0000","last_became_peered":"2024-05-08T03:20:50.783297+0000","last_unstale":"2024-05-08T03:21:16.201450+0000","last_undegraded":"2024-05-08T03:21:16.201450+0000","last_fullsized":"2024-05-08T03:21:16.201450+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:23.915744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":37539555,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":4405,"num_read_kb":30478,"num_write":296,"num_write_kb":15557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":13300321,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900279+0000","last_change":"2024-05-08T03:20:47.821636+0000","last_active":"2024-05-08T03:21:15.900279+0000","last_peered":"2024-05-08T03:21:15.900279+0000","last_clean":"2024-05-08T03:21:15.900279+0000","last_became_active":"2024-05-08T03:20:47.821335+0000","last_became_peered":"2024-05-08T03:20:47.821335+0000","last_unstale":"2024-05-08T03:21:15.900279+0000","last_undegraded":"2024-05-08T03:21:15.900279+0000","last_fullsized":"2024-05-08T03:21:15.900279+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:23.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"115'890","reported_seq":3920,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.207785+0000","last_change":"2024-05-08T03:20:38.806868+0000","last_active":"2024-05-08T03:21:16.207785+0000","last_peered":"2024-05-08T03:21:16.207785+0000","last_clean":"2024-05-08T03:21:16.207785+0000","last_became_active":"2024-05-08T03:20:38.806015+0000","last_became_peered":"2024-05-08T03:20:38.806015+0000","last_unstale":"2024-05-08T03:21:16.207785+0000","last_undegraded":"2024-05-08T03:21:16.207785+0000","last_fullsized":"2024-05-08T03:21:16.207785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:27.452010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":43597160,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":33,"num_read":5435,"num_read_kb":32478,"num_write":1367,"num_write_kb":40057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":44040868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"114'307","reported_seq":3864,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900254+0000","last_change":"2024-05-08T03:20:50.784287+0000","last_active":"2024-05-08T03:21:15.900254+0000","last_peered":"2024-05-08T03:21:15.900254+0000","last_clean":"2024-05-08T03:21:15.900254+0000","last_became_active":"2024-05-08T03:20:50.783285+0000","last_became_peered":"2024-05-08T03:20:50.783285+0000","last_unstale":"2024-05-08T03:21:15.900254+0000","last_undegraded":"2024-05-08T03:21:15.900254+0000","last_fullsized":"2024-05-08T03:21:15.900254+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:04.117886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":22981663,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":2969,"num_read_kb":19685,"num_write":343,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14979418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":355,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903637+0000","last_change":"2024-05-08T03:20:50.784319+0000","last_active":"2024-05-08T03:21:15.903637+0000","last_peered":"2024-05-08T03:21:15.903637+0000","last_clean":"2024-05-08T03:21:15.903637+0000","last_became_active":"2024-05-08T03:20:50.783795+0000","last_became_peered":"2024-05-08T03:20:50.783795+0000","last_unstale":"2024-05-08T03:21:15.903637+0000","last_undegraded":"2024-05-08T03:21:15.903637+0000","last_fullsized":"2024-05-08T03:21:15.903637+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:15.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":210,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903455+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:21:15.903455+0000","last_peered":"2024-05-08T03:21:15.903455+0000","last_clean":"2024-05-08T03:21:15.903455+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:21:15.903455+0000","last_undegraded":"2024-05-08T03:21:15.903455+0000","last_fullsized":"2024-05-08T03:21:15.903455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"115'379","reported_seq":6683,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.222313+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:21:16.222313+0000","last_peered":"2024-05-08T03:21:16.222313+0000","last_clean":"2024-05-08T03:21:16.222313+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:21:16.222313+0000","last_undegraded":"2024-05-08T03:21:16.222313+0000","last_fullsized":"2024-05-08T03:21:16.222313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:39:59.085517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":41472708,"num_objects":89,"num_object_clones":24,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":36,"num_read":6973,"num_read_kb":28509,"num_write":413,"num_write_kb":29416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":78,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900481+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:21:15.900481+0000","last_peered":"2024-05-08T03:21:15.900481+0000","last_clean":"2024-05-08T03:21:15.900481+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:21:15.900481+0000","last_undegraded":"2024-05-08T03:21:15.900481+0000","last_fullsized":"2024-05-08T03:21:15.900481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"115'691","reported_seq":7433,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.201405+0000","last_change":"2024-05-08T03:20:40.781092+0000","last_active":"2024-05-08T03:21:16.201405+0000","last_peered":"2024-05-08T03:21:16.201405+0000","last_clean":"2024-05-08T03:21:16.201405+0000","last_became_active":"2024-05-08T03:20:40.780249+0000","last_became_peered":"2024-05-08T03:20:40.780249+0000","last_unstale":"2024-05-08T03:21:16.201405+0000","last_undegraded":"2024-05-08T03:21:16.201405+0000","last_fullsized":"2024-05-08T03:21:16.201405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.301124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":65338672,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":27,"num_read":10133,"num_read_kb":50423,"num_write":926,"num_write_kb":27860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":211,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.902604+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:21:15.902604+0000","last_peered":"2024-05-08T03:21:15.902604+0000","last_clean":"2024-05-08T03:21:15.902604+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:21:15.902604+0000","last_undegraded":"2024-05-08T03:21:15.902604+0000","last_fullsized":"2024-05-08T03:21:15.902604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"115'748","reported_seq":8305,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900251+0000","last_change":"2024-05-08T03:20:52.800241+0000","last_active":"2024-05-08T03:21:15.900251+0000","last_peered":"2024-05-08T03:21:15.900251+0000","last_clean":"2024-05-08T03:21:15.900251+0000","last_became_active":"2024-05-08T03:20:52.799061+0000","last_became_peered":"2024-05-08T03:20:52.799061+0000","last_unstale":"2024-05-08T03:21:15.900251+0000","last_undegraded":"2024-05-08T03:21:15.900251+0000","last_fullsized":"2024-05-08T03:21:15.900251+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:41.118418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":53322402,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":31,"num_read":9751,"num_read_kb":50954,"num_write":1047,"num_write_kb":16846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903335+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:21:15.903335+0000","last_peered":"2024-05-08T03:21:15.903335+0000","last_clean":"2024-05-08T03:21:15.903335+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:21:15.903335+0000","last_undegraded":"2024-05-08T03:21:15.903335+0000","last_fullsized":"2024-05-08T03:21:15.903335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"115'413","reported_seq":4394,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903362+0000","last_change":"2024-05-08T03:20:51.781338+0000","last_active":"2024-05-08T03:21:15.903362+0000","last_peered":"2024-05-08T03:21:15.903362+0000","last_clean":"2024-05-08T03:21:15.903362+0000","last_became_active":"2024-05-08T03:20:51.780777+0000","last_became_peered":"2024-05-08T03:20:51.780777+0000","last_unstale":"2024-05-08T03:21:15.903362+0000","last_undegraded":"2024-05-08T03:21:15.903362+0000","last_fullsized":"2024-05-08T03:21:15.903362+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:53.155550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":28575026,"num_objects":58,"num_object_clones":10,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":6223,"num_read_kb":16847,"num_write":410,"num_write_kb":7923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5273218,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"116'429","reported_seq":3440,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.027060+0000","last_change":"2024-05-08T03:20:37.784597+0000","last_active":"2024-05-08T03:21:16.027060+0000","last_peered":"2024-05-08T03:21:16.027060+0000","last_clean":"2024-05-08T03:21:16.027060+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:21:16.027060+0000","last_undegraded":"2024-05-08T03:21:16.027060+0000","last_fullsized":"2024-05-08T03:21:16.027060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:50.858166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":90703518,"num_objects":74,"num_object_clones":18,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":32,"num_read":3918,"num_read_kb":26117,"num_write":590,"num_write_kb":21790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"116'466","reported_seq":1818,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900591+0000","last_change":"2024-05-08T03:20:39.776155+0000","last_active":"2024-05-08T03:21:15.900591+0000","last_peered":"2024-05-08T03:21:15.900591+0000","last_clean":"2024-05-08T03:21:15.900591+0000","last_became_active":"2024-05-08T03:20:39.775306+0000","last_became_peered":"2024-05-08T03:20:39.775306+0000","last_unstale":"2024-05-08T03:21:15.900591+0000","last_undegraded":"2024-05-08T03:21:15.900591+0000","last_fullsized":"2024-05-08T03:21:15.900591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:41.452743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":25461244,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":2778,"num_read_kb":10046,"num_write":491,"num_write_kb":5531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"114'352","reported_seq":6691,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900370+0000","last_change":"2024-05-08T03:20:50.784394+0000","last_active":"2024-05-08T03:21:15.900370+0000","last_peered":"2024-05-08T03:21:15.900370+0000","last_clean":"2024-05-08T03:21:15.900370+0000","last_became_active":"2024-05-08T03:20:50.783233+0000","last_became_peered":"2024-05-08T03:20:50.783233+0000","last_unstale":"2024-05-08T03:21:15.900370+0000","last_undegraded":"2024-05-08T03:21:15.900370+0000","last_fullsized":"2024-05-08T03:21:15.900370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:20.673381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":66539048,"num_objects":86,"num_object_clones":21,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":33,"num_read":6173,"num_read_kb":56642,"num_write":406,"num_write_kb":18350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":35463562,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"116'689","reported_seq":4618,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.223611+0000","last_change":"2024-05-08T03:20:52.803353+0000","last_active":"2024-05-08T03:21:16.223611+0000","last_peered":"2024-05-08T03:21:16.223611+0000","last_clean":"2024-05-08T03:21:16.223611+0000","last_became_active":"2024-05-08T03:20:52.802662+0000","last_became_peered":"2024-05-08T03:20:52.802662+0000","last_unstale":"2024-05-08T03:21:16.223611+0000","last_undegraded":"2024-05-08T03:21:16.223611+0000","last_fullsized":"2024-05-08T03:21:16.223611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:00.806749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":88384301,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":36,"num_read":6516,"num_read_kb":22333,"num_write":972,"num_write_kb":23541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"116'854","reported_seq":4145,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900452+0000","last_change":"2024-05-08T03:20:51.785227+0000","last_active":"2024-05-08T03:21:15.900452+0000","last_peered":"2024-05-08T03:21:15.900452+0000","last_clean":"2024-05-08T03:21:15.900452+0000","last_became_active":"2024-05-08T03:20:51.783022+0000","last_became_peered":"2024-05-08T03:20:51.783022+0000","last_unstale":"2024-05-08T03:21:15.900452+0000","last_undegraded":"2024-05-08T03:21:15.900452+0000","last_fullsized":"2024-05-08T03:21:15.900452+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:34.145537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":82668471,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":31,"num_read":7785,"num_read_kb":71032,"num_write":1199,"num_write_kb":48583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":48212116,"num_keys_recovered":59,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"116'387","reported_seq":4050,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.033394+0000","last_change":"2024-05-08T03:20:40.780044+0000","last_active":"2024-05-08T03:21:16.033394+0000","last_peered":"2024-05-08T03:21:16.033394+0000","last_clean":"2024-05-08T03:21:16.033394+0000","last_became_active":"2024-05-08T03:20:40.779118+0000","last_became_peered":"2024-05-08T03:20:40.779118+0000","last_unstale":"2024-05-08T03:21:16.033394+0000","last_undegraded":"2024-05-08T03:21:16.033394+0000","last_fullsized":"2024-05-08T03:21:16.033394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:27.994827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":77537565,"num_objects":88,"num_object_clones":23,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":32,"num_read":3588,"num_read_kb":56016,"num_write":454,"num_write_kb":29337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":51071767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"115'528","reported_seq":3754,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900713+0000","last_change":"2024-05-08T03:20:38.807093+0000","last_active":"2024-05-08T03:21:15.900713+0000","last_peered":"2024-05-08T03:21:15.900713+0000","last_clean":"2024-05-08T03:21:15.900713+0000","last_became_active":"2024-05-08T03:20:38.805719+0000","last_became_peered":"2024-05-08T03:20:38.805719+0000","last_unstale":"2024-05-08T03:21:15.900713+0000","last_undegraded":"2024-05-08T03:21:15.900713+0000","last_fullsized":"2024-05-08T03:21:15.900713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:08:10.441161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":25949934,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":3938,"num_read_kb":15773,"num_write":719,"num_write_kb":13948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":10326037,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"114'382","reported_seq":4311,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.063798+0000","last_change":"2024-05-08T03:20:51.783432+0000","last_active":"2024-05-08T03:21:16.063798+0000","last_peered":"2024-05-08T03:21:16.063798+0000","last_clean":"2024-05-08T03:21:16.063798+0000","last_became_active":"2024-05-08T03:20:51.782569+0000","last_became_peered":"2024-05-08T03:20:51.782569+0000","last_unstale":"2024-05-08T03:21:16.063798+0000","last_undegraded":"2024-05-08T03:21:16.063798+0000","last_fullsized":"2024-05-08T03:21:16.063798+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:10:43.370497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":70131213,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":32,"num_read":5094,"num_read_kb":37466,"num_write":387,"num_write_kb":18326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"115'415","reported_seq":5601,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900256+0000","last_change":"2024-05-08T03:20:38.806539+0000","last_active":"2024-05-08T03:21:15.900256+0000","last_peered":"2024-05-08T03:21:15.900256+0000","last_clean":"2024-05-08T03:21:15.900256+0000","last_became_active":"2024-05-08T03:20:38.805730+0000","last_became_peered":"2024-05-08T03:20:38.805730+0000","last_unstale":"2024-05-08T03:21:15.900256+0000","last_undegraded":"2024-05-08T03:21:15.900256+0000","last_fullsized":"2024-05-08T03:21:15.900256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:02:49.521724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":66655824,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":30,"num_read":6705,"num_read_kb":64838,"num_write":420,"num_write_kb":55014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":60512583,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"116'304","reported_seq":1980,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.223597+0000","last_change":"2024-05-08T03:20:47.825128+0000","last_active":"2024-05-08T03:21:16.223597+0000","last_peered":"2024-05-08T03:21:16.223597+0000","last_clean":"2024-05-08T03:21:16.223597+0000","last_became_active":"2024-05-08T03:20:47.824316+0000","last_became_peered":"2024-05-08T03:20:47.824316+0000","last_unstale":"2024-05-08T03:21:16.223597+0000","last_undegraded":"2024-05-08T03:21:16.223597+0000","last_fullsized":"2024-05-08T03:21:16.223597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:52:47.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":57888817,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":3405,"num_read_kb":27188,"num_write":347,"num_write_kb":29872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"115'362","reported_seq":5088,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900596+0000","last_change":"2024-05-08T03:20:38.806543+0000","last_active":"2024-05-08T03:21:15.900596+0000","last_peered":"2024-05-08T03:21:15.900596+0000","last_clean":"2024-05-08T03:21:15.900596+0000","last_became_active":"2024-05-08T03:20:38.805711+0000","last_became_peered":"2024-05-08T03:20:38.805711+0000","last_unstale":"2024-05-08T03:21:15.900596+0000","last_undegraded":"2024-05-08T03:21:15.900596+0000","last_fullsized":"2024-05-08T03:21:15.900596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:22:12.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":40089528,"num_objects":80,"num_object_clones":22,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":37,"num_read":4863,"num_read_kb":49230,"num_write":509,"num_write_kb":27617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":40436240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1687873749,"num_objects":2258,"num_object_clones":495,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":945,"num_read":173107,"num_read_kb":1135194,"num_write":19962,"num_write_kb":717557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4136,"num_bytes_recovered":610881052,"num_keys_recovered":712,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1228496896,"data_stored":1751386816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15605,"ondisk_log_size":15605,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542172,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268180,"kb_used_data":186244,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94103660,"statfs":{"total":96636764160,"available":96362147840,"internally_reserved":0,"allocated":190713856,"data_stored":275025188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":0,"seq":455266533380,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399936,"kb_used_data":350712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":93971904,"statfs":{"total":96636764160,"available":96227229696,"internally_reserved":0,"allocated":359129088,"data_stored":498689432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542173,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223384,"kb_used_data":174160,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94148456,"statfs":{"total":96636764160,"available":96408018944,"internally_reserved":0,"allocated":178339840,"data_stored":259393315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345344,"kb_used_data":256064,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94026496,"statfs":{"total":96636764160,"available":96283131904,"internally_reserved":0,"allocated":262209536,"data_stored":366916075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542173,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261394432,"data_stored":366793716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177569792,"data_stored":259319361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":135417856,"data_stored":199525057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":358600704,"data_stored":498882218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":189911040,"data_stored":274928069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:21:18.321 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:21:18.321 INFO:teuthology.orchestra.run.smithi177.stdout:5873 trunc from 0x1a3fd85 to 0x84474a2 2024-05-08T03:21:18.321 INFO:teuthology.orchestra.run.smithi177.stdout:5874 clone 46 order 23 su 2097152 sc 14 2024-05-08T03:21:18.415 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:18.414+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:18.460 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:18.460 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:18.460 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:18.460 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:18.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:18.517 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:18.515+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:18.574 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:18.577 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:18.577 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:18.577 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:18.577 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:18.583 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:18.627 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:18.625+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:18.643 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:18.644 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:21:18.655 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-05-08T03:21:18.193453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1688464117,"num_objects":2260,"num_object_clones":495,"num_object_copies":4520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2260,"num_whiteouts":945,"num_read":173223,"num_read_kb":1135294,"num_write":20121,"num_write_kb":719973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":612652156,"num_keys_recovered":712,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15722,"ondisk_log_size":15722,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1236844,"kb_used_data":967180,"kb_used_omap":6,"kb_used_meta":241273,"kb_avail":376250516,"statfs":{"total":386547056640,"available":385280528384,"internally_reserved":0,"allocated":990392320,"data_stored":1400024010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6379,"internal_metadata":247064341},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50412208,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":11,"num_read":12046,"num_read_kb":72572,"num_write":1280,"num_write_kb":66963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000670"},"pg_stats":[{"pgid":"3.1f","version":"112'312","reported_seq":2990,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.902833+0000","last_change":"2024-05-08T03:19:57.058161+0000","last_active":"2024-05-08T03:21:15.902833+0000","last_peered":"2024-05-08T03:21:15.902833+0000","last_clean":"2024-05-08T03:21:15.902833+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:21:15.902833+0000","last_undegraded":"2024-05-08T03:21:15.902833+0000","last_fullsized":"2024-05-08T03:21:15.902833+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:16.621119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010047369999999999,"stat_sum":{"num_bytes":57941618,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":25,"num_read":2874,"num_read_kb":31303,"num_write":365,"num_write_kb":16834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"115'268","reported_seq":2409,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.207417+0000","last_change":"2024-05-08T03:20:50.785288+0000","last_active":"2024-05-08T03:21:16.207417+0000","last_peered":"2024-05-08T03:21:16.207417+0000","last_clean":"2024-05-08T03:21:16.207417+0000","last_became_active":"2024-05-08T03:20:50.783719+0000","last_became_peered":"2024-05-08T03:20:50.783719+0000","last_unstale":"2024-05-08T03:21:16.207417+0000","last_undegraded":"2024-05-08T03:21:16.207417+0000","last_fullsized":"2024-05-08T03:21:16.207417+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:32:30.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079266899999999997,"stat_sum":{"num_bytes":34604698,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":2277,"num_read_kb":7335,"num_write":312,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"116'441","reported_seq":6722,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.044871+0000","last_change":"2024-05-08T03:20:39.774078+0000","last_active":"2024-05-08T03:21:16.044871+0000","last_peered":"2024-05-08T03:21:16.044871+0000","last_clean":"2024-05-08T03:21:16.044871+0000","last_became_active":"2024-05-08T03:20:39.773251+0000","last_became_peered":"2024-05-08T03:20:39.773251+0000","last_unstale":"2024-05-08T03:21:16.044871+0000","last_undegraded":"2024-05-08T03:21:16.044871+0000","last_fullsized":"2024-05-08T03:21:16.044871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:17:20.312838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097278399999999998,"stat_sum":{"num_bytes":72489935,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":20,"num_read":6929,"num_read_kb":44818,"num_write":525,"num_write_kb":14062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"116'879","reported_seq":3781,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.040592+0000","last_change":"2024-05-08T03:20:40.781096+0000","last_active":"2024-05-08T03:21:16.040592+0000","last_peered":"2024-05-08T03:21:16.040592+0000","last_clean":"2024-05-08T03:21:16.040592+0000","last_became_active":"2024-05-08T03:20:40.779478+0000","last_became_peered":"2024-05-08T03:20:40.779478+0000","last_unstale":"2024-05-08T03:21:16.040592+0000","last_undegraded":"2024-05-08T03:21:16.040592+0000","last_fullsized":"2024-05-08T03:21:16.040592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:36:32.730237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082804899999999997,"stat_sum":{"num_bytes":58611786,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":25,"num_read":5504,"num_read_kb":51895,"num_write":1263,"num_write_kb":51140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"116'394","reported_seq":3301,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.108710+0000","last_change":"2024-05-08T03:20:40.780143+0000","last_active":"2024-05-08T03:21:16.108710+0000","last_peered":"2024-05-08T03:21:16.108710+0000","last_clean":"2024-05-08T03:21:16.108710+0000","last_became_active":"2024-05-08T03:20:40.779325+0000","last_became_peered":"2024-05-08T03:20:40.779325+0000","last_unstale":"2024-05-08T03:21:16.108710+0000","last_undegraded":"2024-05-08T03:21:16.108710+0000","last_fullsized":"2024-05-08T03:21:16.108710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:44.350271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054873,"stat_sum":{"num_bytes":42359262,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":29,"num_read":4180,"num_read_kb":21566,"num_write":479,"num_write_kb":13537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7457235,"num_keys_recovered":60,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"115'533","reported_seq":5180,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.200587+0000","last_change":"2024-05-08T03:20:39.775128+0000","last_active":"2024-05-08T03:21:16.200587+0000","last_peered":"2024-05-08T03:21:16.200587+0000","last_clean":"2024-05-08T03:21:16.200587+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:21:16.200587+0000","last_undegraded":"2024-05-08T03:21:16.200587+0000","last_fullsized":"2024-05-08T03:21:16.200587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:12:49.821278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081718800000000005,"stat_sum":{"num_bytes":57834027,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":5450,"num_read_kb":44144,"num_write":737,"num_write_kb":50757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"116'950","reported_seq":3789,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.179503+0000","last_change":"2024-05-08T03:20:51.782224+0000","last_active":"2024-05-08T03:21:16.179503+0000","last_peered":"2024-05-08T03:21:16.179503+0000","last_clean":"2024-05-08T03:21:16.179503+0000","last_became_active":"2024-05-08T03:20:51.780774+0000","last_became_peered":"2024-05-08T03:20:51.780774+0000","last_unstale":"2024-05-08T03:21:16.179503+0000","last_undegraded":"2024-05-08T03:21:16.179503+0000","last_fullsized":"2024-05-08T03:21:16.179503+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:20.051241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060806599999999999,"stat_sum":{"num_bytes":51288125,"num_objects":70,"num_object_clones":11,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":6174,"num_read_kb":22121,"num_write":1534,"num_write_kb":11725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"116'451","reported_seq":8478,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.078690+0000","last_change":"2024-05-08T03:20:50.779322+0000","last_active":"2024-05-08T03:21:16.078690+0000","last_peered":"2024-05-08T03:21:16.078690+0000","last_clean":"2024-05-08T03:21:16.078690+0000","last_became_active":"2024-05-08T03:20:50.777845+0000","last_became_peered":"2024-05-08T03:20:50.777845+0000","last_unstale":"2024-05-08T03:21:16.078690+0000","last_undegraded":"2024-05-08T03:21:16.078690+0000","last_fullsized":"2024-05-08T03:21:16.078690+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:05:06.140301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049891900000000001,"stat_sum":{"num_bytes":77310646,"num_objects":77,"num_object_clones":14,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":33,"num_read":9764,"num_read_kb":57949,"num_write":458,"num_write_kb":15068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":18070131,"num_keys_recovered":60,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"115'329","reported_seq":4276,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903895+0000","last_change":"2024-05-08T03:20:50.781739+0000","last_active":"2024-05-08T03:21:15.903895+0000","last_peered":"2024-05-08T03:21:15.903895+0000","last_clean":"2024-05-08T03:21:15.903895+0000","last_became_active":"2024-05-08T03:20:50.780815+0000","last_became_peered":"2024-05-08T03:20:50.780815+0000","last_unstale":"2024-05-08T03:21:15.903895+0000","last_undegraded":"2024-05-08T03:21:15.903895+0000","last_fullsized":"2024-05-08T03:21:15.903895+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:46.121738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001218954,"stat_sum":{"num_bytes":28956805,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":4659,"num_read_kb":23165,"num_write":347,"num_write_kb":4530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"115'221","reported_seq":3266,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903719+0000","last_change":"2024-05-08T03:20:51.783981+0000","last_active":"2024-05-08T03:21:15.903719+0000","last_peered":"2024-05-08T03:21:15.903719+0000","last_clean":"2024-05-08T03:21:15.903719+0000","last_became_active":"2024-05-08T03:20:51.783155+0000","last_became_peered":"2024-05-08T03:20:51.783155+0000","last_unstale":"2024-05-08T03:21:15.903719+0000","last_undegraded":"2024-05-08T03:21:15.903719+0000","last_fullsized":"2024-05-08T03:21:15.903719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:11:21.253583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085830900000000003,"stat_sum":{"num_bytes":92523192,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":29,"num_read":2596,"num_read_kb":61854,"num_write":213,"num_write_kb":45811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":54103866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"115'292","reported_seq":7014,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900549+0000","last_change":"2024-05-08T03:20:50.785464+0000","last_active":"2024-05-08T03:21:15.900549+0000","last_peered":"2024-05-08T03:21:15.900549+0000","last_clean":"2024-05-08T03:21:15.900549+0000","last_became_active":"2024-05-08T03:20:50.783826+0000","last_became_peered":"2024-05-08T03:20:50.783826+0000","last_unstale":"2024-05-08T03:21:15.900549+0000","last_undegraded":"2024-05-08T03:21:15.900549+0000","last_fullsized":"2024-05-08T03:21:15.900549+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:28.162671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000526937,"stat_sum":{"num_bytes":52883839,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":41,"num_read":6383,"num_read_kb":36940,"num_write":300,"num_write_kb":7491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6817760,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"116'689","reported_seq":4486,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.151059+0000","last_change":"2024-05-08T03:20:37.789834+0000","last_active":"2024-05-08T03:21:16.151059+0000","last_peered":"2024-05-08T03:21:16.151059+0000","last_clean":"2024-05-08T03:21:16.151059+0000","last_became_active":"2024-05-08T03:20:37.788932+0000","last_became_peered":"2024-05-08T03:20:37.788932+0000","last_unstale":"2024-05-08T03:21:16.151059+0000","last_undegraded":"2024-05-08T03:21:16.151059+0000","last_fullsized":"2024-05-08T03:21:16.151059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:34.769709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067012500000000004,"stat_sum":{"num_bytes":31294263,"num_objects":71,"num_object_clones":17,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":5975,"num_read_kb":21317,"num_write":906,"num_write_kb":8259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":2507487,"num_keys_recovered":48,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":52,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.902441+0000","last_change":"2024-05-08T03:20:47.820453+0000","last_active":"2024-05-08T03:21:15.902441+0000","last_peered":"2024-05-08T03:21:15.902441+0000","last_clean":"2024-05-08T03:21:15.902441+0000","last_became_active":"2024-05-08T03:20:47.820180+0000","last_became_peered":"2024-05-08T03:20:47.820180+0000","last_unstale":"2024-05-08T03:21:15.902441+0000","last_undegraded":"2024-05-08T03:21:15.902441+0000","last_fullsized":"2024-05-08T03:21:15.902441+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:33.427981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"116'304","reported_seq":3493,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.117882+0000","last_change":"2024-05-08T03:20:50.782204+0000","last_active":"2024-05-08T03:21:16.117882+0000","last_peered":"2024-05-08T03:21:16.117882+0000","last_clean":"2024-05-08T03:21:16.117882+0000","last_became_active":"2024-05-08T03:20:50.781137+0000","last_became_peered":"2024-05-08T03:20:50.781137+0000","last_unstale":"2024-05-08T03:21:16.117882+0000","last_undegraded":"2024-05-08T03:21:16.117882+0000","last_fullsized":"2024-05-08T03:21:16.117882+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:10:10.778552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010009789999999999,"stat_sum":{"num_bytes":17582148,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":32,"num_read":2984,"num_read_kb":11310,"num_write":331,"num_write_kb":4578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":52,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900614+0000","last_change":"2024-05-08T03:20:47.821931+0000","last_active":"2024-05-08T03:21:15.900614+0000","last_peered":"2024-05-08T03:21:15.900614+0000","last_clean":"2024-05-08T03:21:15.900614+0000","last_became_active":"2024-05-08T03:20:47.821642+0000","last_became_peered":"2024-05-08T03:20:47.821642+0000","last_unstale":"2024-05-08T03:21:15.900614+0000","last_undegraded":"2024-05-08T03:21:15.900614+0000","last_fullsized":"2024-05-08T03:21:15.900614+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:42.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"116'690","reported_seq":4544,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.156222+0000","last_change":"2024-05-08T03:19:22.779096+0000","last_active":"2024-05-08T03:21:16.156222+0000","last_peered":"2024-05-08T03:21:16.156222+0000","last_clean":"2024-05-08T03:21:16.156222+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:21:16.156222+0000","last_undegraded":"2024-05-08T03:21:16.156222+0000","last_fullsized":"2024-05-08T03:21:16.156222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.761270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049802799999999997,"stat_sum":{"num_bytes":27356756,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":30,"num_read":6706,"num_read_kb":33422,"num_write":896,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900639+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:21:15.900639+0000","last_peered":"2024-05-08T03:21:15.900639+0000","last_clean":"2024-05-08T03:21:15.900639+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:21:15.900639+0000","last_undegraded":"2024-05-08T03:21:15.900639+0000","last_fullsized":"2024-05-08T03:21:15.900639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"116'256","reported_seq":5190,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.201450+0000","last_change":"2024-05-08T03:20:50.784522+0000","last_active":"2024-05-08T03:21:16.201450+0000","last_peered":"2024-05-08T03:21:16.201450+0000","last_clean":"2024-05-08T03:21:16.201450+0000","last_became_active":"2024-05-08T03:20:50.783297+0000","last_became_peered":"2024-05-08T03:20:50.783297+0000","last_unstale":"2024-05-08T03:21:16.201450+0000","last_undegraded":"2024-05-08T03:21:16.201450+0000","last_fullsized":"2024-05-08T03:21:16.201450+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:23.915744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076354600000000004,"stat_sum":{"num_bytes":37539555,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":4405,"num_read_kb":30478,"num_write":296,"num_write_kb":15557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":13300321,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900279+0000","last_change":"2024-05-08T03:20:47.821636+0000","last_active":"2024-05-08T03:21:15.900279+0000","last_peered":"2024-05-08T03:21:15.900279+0000","last_clean":"2024-05-08T03:21:15.900279+0000","last_became_active":"2024-05-08T03:20:47.821335+0000","last_became_peered":"2024-05-08T03:20:47.821335+0000","last_unstale":"2024-05-08T03:21:15.900279+0000","last_undegraded":"2024-05-08T03:21:15.900279+0000","last_fullsized":"2024-05-08T03:21:15.900279+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:23.980484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"115'890","reported_seq":3920,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.207785+0000","last_change":"2024-05-08T03:20:38.806868+0000","last_active":"2024-05-08T03:21:16.207785+0000","last_peered":"2024-05-08T03:21:16.207785+0000","last_clean":"2024-05-08T03:21:16.207785+0000","last_became_active":"2024-05-08T03:20:38.806015+0000","last_became_peered":"2024-05-08T03:20:38.806015+0000","last_unstale":"2024-05-08T03:21:16.207785+0000","last_undegraded":"2024-05-08T03:21:16.207785+0000","last_fullsized":"2024-05-08T03:21:16.207785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:27.452010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276199999999999,"stat_sum":{"num_bytes":43597160,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":33,"num_read":5435,"num_read_kb":32478,"num_write":1367,"num_write_kb":40057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":44040868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"114'307","reported_seq":3864,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900254+0000","last_change":"2024-05-08T03:20:50.784287+0000","last_active":"2024-05-08T03:21:15.900254+0000","last_peered":"2024-05-08T03:21:15.900254+0000","last_clean":"2024-05-08T03:21:15.900254+0000","last_became_active":"2024-05-08T03:20:50.783285+0000","last_became_peered":"2024-05-08T03:20:50.783285+0000","last_unstale":"2024-05-08T03:21:15.900254+0000","last_undegraded":"2024-05-08T03:21:15.900254+0000","last_fullsized":"2024-05-08T03:21:15.900254+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:04.117886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041209800000000001,"stat_sum":{"num_bytes":22981663,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":2969,"num_read_kb":19685,"num_write":343,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14979418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":355,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903637+0000","last_change":"2024-05-08T03:20:50.784319+0000","last_active":"2024-05-08T03:21:15.903637+0000","last_peered":"2024-05-08T03:21:15.903637+0000","last_clean":"2024-05-08T03:21:15.903637+0000","last_became_active":"2024-05-08T03:20:50.783795+0000","last_became_peered":"2024-05-08T03:20:50.783795+0000","last_unstale":"2024-05-08T03:21:15.903637+0000","last_undegraded":"2024-05-08T03:21:15.903637+0000","last_fullsized":"2024-05-08T03:21:15.903637+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:15.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":210,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903455+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:21:15.903455+0000","last_peered":"2024-05-08T03:21:15.903455+0000","last_clean":"2024-05-08T03:21:15.903455+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:21:15.903455+0000","last_undegraded":"2024-05-08T03:21:15.903455+0000","last_fullsized":"2024-05-08T03:21:15.903455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"115'379","reported_seq":6683,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.222313+0000","last_change":"2024-05-08T03:19:22.779313+0000","last_active":"2024-05-08T03:21:16.222313+0000","last_peered":"2024-05-08T03:21:16.222313+0000","last_clean":"2024-05-08T03:21:16.222313+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:21:16.222313+0000","last_undegraded":"2024-05-08T03:21:16.222313+0000","last_fullsized":"2024-05-08T03:21:16.222313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:39:59.085517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059418899999999996,"stat_sum":{"num_bytes":41472708,"num_objects":89,"num_object_clones":24,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":36,"num_read":6973,"num_read_kb":28509,"num_write":413,"num_write_kb":29416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":78,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900481+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:21:15.900481+0000","last_peered":"2024-05-08T03:21:15.900481+0000","last_clean":"2024-05-08T03:21:15.900481+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:21:15.900481+0000","last_undegraded":"2024-05-08T03:21:15.900481+0000","last_fullsized":"2024-05-08T03:21:15.900481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"115'691","reported_seq":7433,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.201405+0000","last_change":"2024-05-08T03:20:40.781092+0000","last_active":"2024-05-08T03:21:16.201405+0000","last_peered":"2024-05-08T03:21:16.201405+0000","last_clean":"2024-05-08T03:21:16.201405+0000","last_became_active":"2024-05-08T03:20:40.780249+0000","last_became_peered":"2024-05-08T03:20:40.780249+0000","last_unstale":"2024-05-08T03:21:16.201405+0000","last_undegraded":"2024-05-08T03:21:16.201405+0000","last_fullsized":"2024-05-08T03:21:16.201405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.301124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048572600000000002,"stat_sum":{"num_bytes":65338672,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":27,"num_read":10133,"num_read_kb":50423,"num_write":926,"num_write_kb":27860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":211,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.902604+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:21:15.902604+0000","last_peered":"2024-05-08T03:21:15.902604+0000","last_clean":"2024-05-08T03:21:15.902604+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:21:15.902604+0000","last_undegraded":"2024-05-08T03:21:15.902604+0000","last_fullsized":"2024-05-08T03:21:15.902604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"115'748","reported_seq":8305,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900251+0000","last_change":"2024-05-08T03:20:52.800241+0000","last_active":"2024-05-08T03:21:15.900251+0000","last_peered":"2024-05-08T03:21:15.900251+0000","last_clean":"2024-05-08T03:21:15.900251+0000","last_became_active":"2024-05-08T03:20:52.799061+0000","last_became_peered":"2024-05-08T03:20:52.799061+0000","last_unstale":"2024-05-08T03:21:15.900251+0000","last_undegraded":"2024-05-08T03:21:15.900251+0000","last_fullsized":"2024-05-08T03:21:15.900251+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:41.118418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001086044,"stat_sum":{"num_bytes":53322402,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":31,"num_read":9751,"num_read_kb":50954,"num_write":1047,"num_write_kb":16846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903335+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:21:15.903335+0000","last_peered":"2024-05-08T03:21:15.903335+0000","last_clean":"2024-05-08T03:21:15.903335+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:21:15.903335+0000","last_undegraded":"2024-05-08T03:21:15.903335+0000","last_fullsized":"2024-05-08T03:21:15.903335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"115'413","reported_seq":4394,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.903362+0000","last_change":"2024-05-08T03:20:51.781338+0000","last_active":"2024-05-08T03:21:15.903362+0000","last_peered":"2024-05-08T03:21:15.903362+0000","last_clean":"2024-05-08T03:21:15.903362+0000","last_became_active":"2024-05-08T03:20:51.780777+0000","last_became_peered":"2024-05-08T03:20:51.780777+0000","last_unstale":"2024-05-08T03:21:15.903362+0000","last_undegraded":"2024-05-08T03:21:15.903362+0000","last_fullsized":"2024-05-08T03:21:15.903362+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:53.155550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086960900000000003,"stat_sum":{"num_bytes":28575026,"num_objects":58,"num_object_clones":10,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":6223,"num_read_kb":16847,"num_write":410,"num_write_kb":7923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5273218,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"116'429","reported_seq":3440,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.027060+0000","last_change":"2024-05-08T03:20:37.784597+0000","last_active":"2024-05-08T03:21:16.027060+0000","last_peered":"2024-05-08T03:21:16.027060+0000","last_clean":"2024-05-08T03:21:16.027060+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:21:16.027060+0000","last_undegraded":"2024-05-08T03:21:16.027060+0000","last_fullsized":"2024-05-08T03:21:16.027060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:50.858166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034910300000000002,"stat_sum":{"num_bytes":90703518,"num_objects":74,"num_object_clones":18,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":32,"num_read":3918,"num_read_kb":26117,"num_write":590,"num_write_kb":21790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"116'466","reported_seq":1818,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900591+0000","last_change":"2024-05-08T03:20:39.776155+0000","last_active":"2024-05-08T03:21:15.900591+0000","last_peered":"2024-05-08T03:21:15.900591+0000","last_clean":"2024-05-08T03:21:15.900591+0000","last_became_active":"2024-05-08T03:20:39.775306+0000","last_became_peered":"2024-05-08T03:20:39.775306+0000","last_unstale":"2024-05-08T03:21:15.900591+0000","last_undegraded":"2024-05-08T03:21:15.900591+0000","last_fullsized":"2024-05-08T03:21:15.900591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:41.452743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026601899999999999,"stat_sum":{"num_bytes":25461244,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":2778,"num_read_kb":10046,"num_write":491,"num_write_kb":5531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"114'352","reported_seq":6691,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900370+0000","last_change":"2024-05-08T03:20:50.784394+0000","last_active":"2024-05-08T03:21:15.900370+0000","last_peered":"2024-05-08T03:21:15.900370+0000","last_clean":"2024-05-08T03:21:15.900370+0000","last_became_active":"2024-05-08T03:20:50.783233+0000","last_became_peered":"2024-05-08T03:20:50.783233+0000","last_unstale":"2024-05-08T03:21:15.900370+0000","last_undegraded":"2024-05-08T03:21:15.900370+0000","last_fullsized":"2024-05-08T03:21:15.900370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:20.673381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050955499999999997,"stat_sum":{"num_bytes":66539048,"num_objects":86,"num_object_clones":21,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":33,"num_read":6173,"num_read_kb":56642,"num_write":406,"num_write_kb":18350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":35463562,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"116'689","reported_seq":4618,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.223611+0000","last_change":"2024-05-08T03:20:52.803353+0000","last_active":"2024-05-08T03:21:16.223611+0000","last_peered":"2024-05-08T03:21:16.223611+0000","last_clean":"2024-05-08T03:21:16.223611+0000","last_became_active":"2024-05-08T03:20:52.802662+0000","last_became_peered":"2024-05-08T03:20:52.802662+0000","last_unstale":"2024-05-08T03:21:16.223611+0000","last_undegraded":"2024-05-08T03:21:16.223611+0000","last_fullsized":"2024-05-08T03:21:16.223611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:00.806749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037018400000000002,"stat_sum":{"num_bytes":88384301,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":36,"num_read":6516,"num_read_kb":22333,"num_write":972,"num_write_kb":23541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"116'854","reported_seq":4145,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900452+0000","last_change":"2024-05-08T03:20:51.785227+0000","last_active":"2024-05-08T03:21:15.900452+0000","last_peered":"2024-05-08T03:21:15.900452+0000","last_clean":"2024-05-08T03:21:15.900452+0000","last_became_active":"2024-05-08T03:20:51.783022+0000","last_became_peered":"2024-05-08T03:20:51.783022+0000","last_unstale":"2024-05-08T03:21:15.900452+0000","last_undegraded":"2024-05-08T03:21:15.900452+0000","last_fullsized":"2024-05-08T03:21:15.900452+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:34.145537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000509419,"stat_sum":{"num_bytes":82668471,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":31,"num_read":7785,"num_read_kb":71032,"num_write":1199,"num_write_kb":48583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":48212116,"num_keys_recovered":59,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"116'387","reported_seq":4050,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.033394+0000","last_change":"2024-05-08T03:20:40.780044+0000","last_active":"2024-05-08T03:21:16.033394+0000","last_peered":"2024-05-08T03:21:16.033394+0000","last_clean":"2024-05-08T03:21:16.033394+0000","last_became_active":"2024-05-08T03:20:40.779118+0000","last_became_peered":"2024-05-08T03:20:40.779118+0000","last_unstale":"2024-05-08T03:21:16.033394+0000","last_undegraded":"2024-05-08T03:21:16.033394+0000","last_fullsized":"2024-05-08T03:21:16.033394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:28:27.994827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061641300000000001,"stat_sum":{"num_bytes":77537565,"num_objects":88,"num_object_clones":23,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":32,"num_read":3588,"num_read_kb":56016,"num_write":454,"num_write_kb":29337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":51071767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"115'528","reported_seq":3754,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900713+0000","last_change":"2024-05-08T03:20:38.807093+0000","last_active":"2024-05-08T03:21:15.900713+0000","last_peered":"2024-05-08T03:21:15.900713+0000","last_clean":"2024-05-08T03:21:15.900713+0000","last_became_active":"2024-05-08T03:20:38.805719+0000","last_became_peered":"2024-05-08T03:20:38.805719+0000","last_unstale":"2024-05-08T03:21:15.900713+0000","last_undegraded":"2024-05-08T03:21:15.900713+0000","last_fullsized":"2024-05-08T03:21:15.900713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:08:10.441161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056539600000000002,"stat_sum":{"num_bytes":25949934,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":3938,"num_read_kb":15773,"num_write":719,"num_write_kb":13948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":10326037,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"114'382","reported_seq":4311,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.063798+0000","last_change":"2024-05-08T03:20:51.783432+0000","last_active":"2024-05-08T03:21:16.063798+0000","last_peered":"2024-05-08T03:21:16.063798+0000","last_clean":"2024-05-08T03:21:16.063798+0000","last_became_active":"2024-05-08T03:20:51.782569+0000","last_became_peered":"2024-05-08T03:20:51.782569+0000","last_unstale":"2024-05-08T03:21:16.063798+0000","last_undegraded":"2024-05-08T03:21:16.063798+0000","last_fullsized":"2024-05-08T03:21:16.063798+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:10:43.370497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026123599999999998,"stat_sum":{"num_bytes":70131213,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":32,"num_read":5094,"num_read_kb":37466,"num_write":387,"num_write_kb":18326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"115'415","reported_seq":5601,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900256+0000","last_change":"2024-05-08T03:20:38.806539+0000","last_active":"2024-05-08T03:21:15.900256+0000","last_peered":"2024-05-08T03:21:15.900256+0000","last_clean":"2024-05-08T03:21:15.900256+0000","last_became_active":"2024-05-08T03:20:38.805730+0000","last_became_peered":"2024-05-08T03:20:38.805730+0000","last_unstale":"2024-05-08T03:21:15.900256+0000","last_undegraded":"2024-05-08T03:21:15.900256+0000","last_fullsized":"2024-05-08T03:21:15.900256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:02:49.521724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039548200000000001,"stat_sum":{"num_bytes":66655824,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":30,"num_read":6705,"num_read_kb":64838,"num_write":420,"num_write_kb":55014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":60512583,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"116'304","reported_seq":1980,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:16.223597+0000","last_change":"2024-05-08T03:20:47.825128+0000","last_active":"2024-05-08T03:21:16.223597+0000","last_peered":"2024-05-08T03:21:16.223597+0000","last_clean":"2024-05-08T03:21:16.223597+0000","last_became_active":"2024-05-08T03:20:47.824316+0000","last_became_peered":"2024-05-08T03:20:47.824316+0000","last_unstale":"2024-05-08T03:21:16.223597+0000","last_undegraded":"2024-05-08T03:21:16.223597+0000","last_fullsized":"2024-05-08T03:21:16.223597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:52:47.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015312,"stat_sum":{"num_bytes":57888817,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":3405,"num_read_kb":27188,"num_write":347,"num_write_kb":29872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"115'362","reported_seq":5088,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-08T03:21:15.900596+0000","last_change":"2024-05-08T03:20:38.806543+0000","last_active":"2024-05-08T03:21:15.900596+0000","last_peered":"2024-05-08T03:21:15.900596+0000","last_clean":"2024-05-08T03:21:15.900596+0000","last_became_active":"2024-05-08T03:20:38.805711+0000","last_became_peered":"2024-05-08T03:20:38.805711+0000","last_unstale":"2024-05-08T03:21:15.900596+0000","last_undegraded":"2024-05-08T03:21:15.900596+0000","last_fullsized":"2024-05-08T03:21:15.900596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:22:12.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036317499999999998,"stat_sum":{"num_bytes":40089528,"num_objects":80,"num_object_clones":22,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":37,"num_read":4863,"num_read_kb":49230,"num_write":509,"num_write_kb":27617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":40436240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1687873749,"num_objects":2258,"num_object_clones":495,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":945,"num_read":173107,"num_read_kb":1135194,"num_write":19962,"num_write_kb":717557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4136,"num_bytes_recovered":610881052,"num_keys_recovered":712,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1228496896,"data_stored":1751386816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15605,"ondisk_log_size":15605,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542172,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268180,"kb_used_data":186244,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94103660,"statfs":{"total":96636764160,"available":96362147840,"internally_reserved":0,"allocated":190713856,"data_stored":275025188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":0,"seq":455266533380,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399936,"kb_used_data":350712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":93971904,"statfs":{"total":96636764160,"available":96227229696,"internally_reserved":0,"allocated":359129088,"data_stored":498689432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542173,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223384,"kb_used_data":174160,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94148456,"statfs":{"total":96636764160,"available":96408018944,"internally_reserved":0,"allocated":178339840,"data_stored":259393315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345344,"kb_used_data":256064,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94026496,"statfs":{"total":96636764160,"available":96283131904,"internally_reserved":0,"allocated":262209536,"data_stored":366916075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542173,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261394432,"data_stored":366793716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177569792,"data_stored":259319361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":135417856,"data_stored":199525057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":358600704,"data_stored":498882218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":189911040,"data_stored":274928069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:21:18.657 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T03:21:18.657 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:18.657 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T03:21:18.658 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.531394 2024-05-08T03:21:18.658 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.531393536361983 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 106.812949852, 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:18.689 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.690 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.691 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:18.692 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.693 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:18.694 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47100866, 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.695 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:18.696 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.697 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.698 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470993261, 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:18.699 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.700 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:18.701 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.702 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47096161899999, 2024-05-08T03:21:18.703 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.704 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.705 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:18.706 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.707 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470948287, 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.708 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:18.709 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:18.710 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.711 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47093849300001, 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.712 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:18.713 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:18.714 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.715 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47092908, 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.716 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.717 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.718 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470919559, 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:18.719 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.720 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:18.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:18.722 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470910218, 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:18.723 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.724 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:18.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:18.726 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.727 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47090100299999, 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:18.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:18.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.731 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47089164099999, 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:18.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.734 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.735 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470882209, 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.736 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.737 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:18.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:18.739 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470872793, 2024-05-08T03:21:18.740 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:18.742 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:18.744 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470863775, 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:18.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47085473200001, 2024-05-08T03:21:18.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:18.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47084243899999, 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:18.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470833172, 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.757 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:18.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:18.760 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.470819314, 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.761 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:18.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.764 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.47081002100001, 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:18.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 102.477795888, 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:18.769 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.770 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:18.771 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:18.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:18.773 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:18.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:18.775 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:18.773+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:18.875 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:18.873+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:18.923 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:18.923 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:18.924 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:18.924 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:18.927 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:21:18.933 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:18.977 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:18.975+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:18.983 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone45 from 0x84474a2 (overlap 0x1a3fd85) to 0xfbfe83 2024-05-08T03:21:19.046 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-05-08T03:21:19.050 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:19.050 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:19.050 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:19.050 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:19.051 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:19.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:19.077 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:19.075+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:19.178 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:19.176+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:19.207 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:19.207 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:19.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:19.207 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:19.207 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:19.216 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:19.250 INFO:teuthology.orchestra.run.smithi177.stdout:5875 punch from 0x57bd7b1 to 0x57c1846, (0x4095 bytes) 2024-05-08T03:21:19.262 INFO:teuthology.orchestra.run.smithi177.stdout:5876 trunc from 0x84474a2 to 0x2e36be3 2024-05-08T03:21:19.279 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:19.277+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:19.294 INFO:teuthology.orchestra.run.smithi177.stdout:5877 read 0x2091cac thru 0x209f289 (0xd5de bytes) 2024-05-08T03:21:19.296 INFO:teuthology.orchestra.run.smithi177.stdout:5878 trunc from 0x2e36be3 to 0x187c32d 2024-05-08T03:21:19.317 INFO:teuthology.orchestra.run.smithi177.stdout:5881 writesame 0x27ee940 thru 0x27f4f66 data_size 0x17(0x6627 bytes) 2024-05-08T03:21:19.318 INFO:teuthology.orchestra.run.smithi177.stdout:5883 punch from 0x52768f to 0x52b60b, (0x3f7c bytes) 2024-05-08T03:21:19.318 INFO:teuthology.orchestra.run.smithi177.stdout:5885 write 0x170798c thru 0x1714e04 (0xd479 bytes) 2024-05-08T03:21:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:5887 punch from 0x1d686cf to 0x1d6c85f, (0x4190 bytes) 2024-05-08T03:21:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:5889 punch from 0x1ba6c7 to 0x1c30cb, (0x8a04 bytes) 2024-05-08T03:21:19.321 INFO:teuthology.orchestra.run.smithi177.stdout:5890 punch from 0x2229f55 to 0x2235caa, (0xbd55 bytes) 2024-05-08T03:21:19.338 INFO:teuthology.orchestra.run.smithi177.stdout:5891 writesame 0xcac10fe thru 0xcad09d0 data_size 0x11(0xf8d3 bytes) 2024-05-08T03:21:19.338 INFO:teuthology.orchestra.run.smithi177.stdout:5893 write 0x218a594 thru 0x218f186 (0x4bf3 bytes) 2024-05-08T03:21:19.338 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:19.339 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:19.339 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:19.339 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:19.342 INFO:teuthology.orchestra.run.smithi177.stdout:5894 trunc from 0xcad09d1 to 0xaa49e49 2024-05-08T03:21:19.347 INFO:teuthology.orchestra.run.smithi177.stdout:5896 punch from 0x73da1b0 to 0x73de90b, (0x475b bytes) 2024-05-08T03:21:19.347 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:19.347 INFO:teuthology.orchestra.run.smithi177.stdout:5898 punch from 0x4d08320 to 0x4d14700, (0xc3e0 bytes) 2024-05-08T03:21:19.347 INFO:teuthology.orchestra.run.smithi177.stdout:5900 write 0x119968c thru 0x11a11ce (0x7b43 bytes) 2024-05-08T03:21:19.349 INFO:teuthology.orchestra.run.smithi177.stdout:5902 read 0x8dda48e thru 0x8de76eb (0xd25e bytes) 2024-05-08T03:21:19.349 INFO:teuthology.orchestra.run.smithi177.stdout:5903 read 0xa183bb8 thru 0xa18b6ab (0x7af4 bytes) 2024-05-08T03:21:19.349 INFO:teuthology.orchestra.run.smithi177.stdout:5905 read 0x5e5ce08 thru 0x5e6cca3 (0xfe9c bytes) 2024-05-08T03:21:19.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:19.351 INFO:teuthology.orchestra.run.smithi177.stdout:5907 punch from 0x352262 to 0x35c971, (0xa70f bytes) 2024-05-08T03:21:19.351 INFO:teuthology.orchestra.run.smithi177.stdout:5908 write 0x18796c4 thru 0x1881d8f (0x86cc bytes) 2024-05-08T03:21:19.352 INFO:teuthology.orchestra.run.smithi177.stdout:5909 read 0x17d04d1 thru 0x17de164 (0xdc94 bytes) 2024-05-08T03:21:19.352 INFO:teuthology.orchestra.run.smithi177.stdout:5910 read 0x5c15445 thru 0x5c156fd (0x2b9 bytes) 2024-05-08T03:21:19.352 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:19.360 INFO:teuthology.orchestra.run.smithi177.stdout:5912 trunc from 0xaa49e49 to 0x34b0a50 2024-05-08T03:21:19.365 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:19.380 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:19.378+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:19.395 INFO:teuthology.orchestra.run.smithi177.stdout:5913 writesame 0xe6918b0 thru 0xe6971ec data_size 0x5(0x593d bytes) 2024-05-08T03:21:19.395 INFO:teuthology.orchestra.run.smithi177.stdout:5914 write 0x34a8334 thru 0x34aed7c (0x6a49 bytes) 2024-05-08T03:21:19.399 INFO:teuthology.orchestra.run.smithi177.stdout:5915 punch from 0x91bc044 to 0x91ca79e, (0xe75a bytes) 2024-05-08T03:21:19.400 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:19.400 INFO:teuthology.orchestra.run.smithi177.stdout:5917 punch from 0x8c6ce1a to 0x8c703e1, (0x35c7 bytes) 2024-05-08T03:21:19.407 INFO:teuthology.orchestra.run.smithi177.stdout:5919 write 0xe85e5d7 thru 0xe86bfb5 (0xd9df bytes) 2024-05-08T03:21:19.410 INFO:teuthology.orchestra.run.smithi177.stdout:5920 read 0xa20792c thru 0xa208f86 (0x165b bytes) 2024-05-08T03:21:19.411 INFO:teuthology.orchestra.run.smithi177.stdout:5921 read 0x7c2fcf0 thru 0x7c30ef2 (0x1203 bytes) 2024-05-08T03:21:19.411 INFO:teuthology.orchestra.run.smithi177.stdout:5922 punch from 0xf8964a to 0xf8a0f4, (0xaaa bytes) 2024-05-08T03:21:19.411 INFO:teuthology.orchestra.run.smithi177.stdout:5923 write 0xc83a4ff thru 0xc8405b7 (0x60b9 bytes) 2024-05-08T03:21:19.413 INFO:teuthology.orchestra.run.smithi177.stdout:5924 write 0x3ca0f2b thru 0x3cab5db (0xa6b1 bytes) 2024-05-08T03:21:19.415 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:19.430 INFO:teuthology.orchestra.run.smithi177.stdout:5928 trunc from 0xe86bfb6 to 0x19a2ff5 2024-05-08T03:21:19.448 INFO:teuthology.orchestra.run.smithi177.stdout:5929 punch from 0x19bcc7 to 0x19bcc8, (0x1 bytes) 2024-05-08T03:21:19.448 INFO:teuthology.orchestra.run.smithi177.stdout:5930 writesame 0x15e57a4 thru 0x15f5222 data_size 0x1(0xfa7f bytes) 2024-05-08T03:21:19.471 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:19.471 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:19.471 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:19.471 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:19.471 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:19.479 INFO:teuthology.orchestra.run.smithi177.stdout:5931 read 0x11ea9e7 thru 0x11ef21b (0x4835 bytes) 2024-05-08T03:21:19.480 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:19.479+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:19.481 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:19.497 INFO:teuthology.orchestra.run.smithi177.stdout:5932 write 0xacb01e1 thru 0xacbb523 (0xb343 bytes) 2024-05-08T03:21:19.500 INFO:teuthology.orchestra.run.smithi177.stdout:5936 read 0x5e74fae thru 0x5e7674a (0x179d bytes) 2024-05-08T03:21:19.500 INFO:teuthology.orchestra.run.smithi177.stdout:5937 write 0x9b740c6 thru 0x9b78f1d (0x4e58 bytes) 2024-05-08T03:21:19.502 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:19.502 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:19.504 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:19.504 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:19.504 INFO:teuthology.orchestra.run.smithi177.stdout:5941 writesame 0x694b8e9 thru 0x6958400 data_size 0x8(0xcb18 bytes) 2024-05-08T03:21:19.508 INFO:teuthology.orchestra.run.smithi177.stdout:5942 punch from 0x9a5f9be to 0x9a6d479, (0xdabb bytes) 2024-05-08T03:21:19.508 INFO:teuthology.orchestra.run.smithi177.stdout:5944 punch from 0x651fa39 to 0x6527bf0, (0x81b7 bytes) 2024-05-08T03:21:19.508 INFO:teuthology.orchestra.run.smithi177.stdout:5945 clone 47 order 20 su 8192 sc 10 2024-05-08T03:21:19.522 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:19.523 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:19.540 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:19.540 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:19.557 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:19.557 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:19.574 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:19.574 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:19.582 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:19.580+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:19.591 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:19.591 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:19.604 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:19.604 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:19.609 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:19.609 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:19.614 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:19.626 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:19.627 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:19.683 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:19.681+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:19.731 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:19.731 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:19.740 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:19.784 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:19.782+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:19.884 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:19.883+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:19.895 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:19.920 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:21:19.921 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.4 primary-affinity to 0.531394 (8348252) 2024-05-08T03:21:19.986 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:19.984+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:20.028 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.028 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.037 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:20.087 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:20.085+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:20.144 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.144 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.152 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:20.188 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:20.186+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:20.289 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:20.288+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:20.297 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.297 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.306 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:20.391 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:20.389+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:20.411 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.411 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.420 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:20.491 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:20.490+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:20.526 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.526 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.535 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:20.592 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:20.591+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:20.638 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.638 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.647 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:20.693 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:20.692+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:20.753 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.753 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.762 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:20.807 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:20.805+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:20.879 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:20.879 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:20.888 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:20.929 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:20.927+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:20.986 INFO:teuthology.orchestra.run.smithi177.stdout:truncating image image_client.0-clone46 from 0xacbb524 (overlap 0x187c32d) to 0x160960e 2024-05-08T03:21:20.995 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:20.994+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:21.042 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.042 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:21.080 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-05-08T03:21:21.096 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:21.095+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:21.102 INFO:teuthology.orchestra.run.smithi177.stdout:5946 trunc from 0xacbb524 to 0x984f682 2024-05-08T03:21:21.169 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.169 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.169 INFO:teuthology.orchestra.run.smithi177.stdout:5947 write 0xe057ced thru 0xe06749b (0xf7af bytes) 2024-05-08T03:21:21.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:21.183 INFO:teuthology.orchestra.run.smithi177.stdout:5948 read 0x997f05e thru 0x998d108 (0xe0ab bytes) 2024-05-08T03:21:21.184 INFO:teuthology.orchestra.run.smithi177.stdout:5949 write 0x7a79204 thru 0x7a84844 (0xb641 bytes) 2024-05-08T03:21:21.197 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:21.196+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:21.215 INFO:teuthology.orchestra.run.smithi177.stdout:5950 write 0x378398a thru 0x37908da (0xcf51 bytes) 2024-05-08T03:21:21.259 INFO:teuthology.orchestra.run.smithi177.stdout:5951 read 0x7ef0f11 thru 0x7efeca3 (0xdd93 bytes) 2024-05-08T03:21:21.260 INFO:teuthology.orchestra.run.smithi177.stdout:5952 read 0x75bb898 thru 0x75cafb7 (0xf720 bytes) 2024-05-08T03:21:21.262 INFO:teuthology.orchestra.run.smithi177.stdout:5953 write 0xa50faea thru 0xa51bd72 (0xc289 bytes) 2024-05-08T03:21:21.273 INFO:teuthology.orchestra.run.smithi177.stdout:5954 read 0x235512c thru 0x2355555 (0x42a bytes) 2024-05-08T03:21:21.277 INFO:teuthology.orchestra.run.smithi177.stdout:5955 trunc from 0xe06749c to 0xb0e5091 2024-05-08T03:21:21.294 INFO:teuthology.orchestra.run.smithi177.stdout:5956 trunc from 0xb0e5091 to 0xca150e4 2024-05-08T03:21:21.298 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:21.296+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:21.300 INFO:teuthology.orchestra.run.smithi177.stdout:5957 writesame 0xb94bc30 thru 0xb9533bf data_size 0x8(0x7790 bytes) 2024-05-08T03:21:21.300 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.300 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.311 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:21.319 INFO:teuthology.orchestra.run.smithi177.stdout:5958 trunc from 0xca150e4 to 0x2942bab 2024-05-08T03:21:21.399 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:21.397+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:21.411 INFO:teuthology.orchestra.run.smithi177.stdout:5960 trunc from 0x2942bab to 0x1fba106 2024-05-08T03:21:21.431 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.431 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:21.501 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:21.499+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:21.508 INFO:teuthology.orchestra.run.smithi177.stdout:5961 punch from 0x1e586ce to 0x1e59d84, (0x16b6 bytes) 2024-05-08T03:21:21.508 INFO:teuthology.orchestra.run.smithi177.stdout:5962 read 0x1239fcd thru 0x12453ff (0xb433 bytes) 2024-05-08T03:21:21.522 INFO:teuthology.orchestra.run.smithi177.stdout:5963 trunc from 0x1fba106 to 0xdb5d51b 2024-05-08T03:21:21.528 INFO:teuthology.orchestra.run.smithi177.stdout:5964 read 0x57fd2f5 thru 0x58084b0 (0xb1bc bytes) 2024-05-08T03:21:21.534 INFO:teuthology.orchestra.run.smithi177.stdout:5966 write 0xe3ebf22 thru 0xe3f4218 (0x82f7 bytes) 2024-05-08T03:21:21.543 INFO:teuthology.orchestra.run.smithi177.stdout:5967 write 0x7ea7f9d thru 0x7eaaa37 (0x2a9b bytes) 2024-05-08T03:21:21.548 INFO:teuthology.orchestra.run.smithi177.stdout:5968 read 0x3a91634 thru 0x3a9326a (0x1c37 bytes) 2024-05-08T03:21:21.548 INFO:teuthology.orchestra.run.smithi177.stdout:5969 read 0x486a8b0 thru 0x4874eed (0xa63e bytes) 2024-05-08T03:21:21.548 INFO:teuthology.orchestra.run.smithi177.stdout:5970 punch from 0x90e7263 to 0x90ecc89, (0x5a26 bytes) 2024-05-08T03:21:21.548 INFO:teuthology.orchestra.run.smithi177.stdout:compare and write disabled 2024-05-08T03:21:21.548 INFO:teuthology.orchestra.run.smithi177.stdout:5974 read 0x6472fbc thru 0x6481db1 (0xedf6 bytes) 2024-05-08T03:21:21.549 INFO:teuthology.orchestra.run.smithi177.stdout:5976 write 0x3fef242 thru 0x3ff23e3 (0x31a2 bytes) 2024-05-08T03:21:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.560 INFO:teuthology.orchestra.run.smithi177.stdout:5977 trunc from 0xe3f4219 to 0x858dbd3 2024-05-08T03:21:21.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:21.571 INFO:teuthology.orchestra.run.smithi177.stdout:5978 read 0x54ebd56 thru 0x54ef3ba (0x3665 bytes) 2024-05-08T03:21:21.571 INFO:teuthology.orchestra.run.smithi177.stdout:5979 read 0x2ce056f thru 0x2ce696a (0x63fc bytes) 2024-05-08T03:21:21.571 INFO:teuthology.orchestra.run.smithi177.stdout:5980 read 0x2d00f78 thru 0x2d08d33 (0x7dbc bytes) 2024-05-08T03:21:21.571 INFO:teuthology.orchestra.run.smithi177.stdout:5981 flatten 2024-05-08T03:21:21.606 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:21.604+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:21.638 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:21.687 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.687 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.699 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:21.740 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:21.738+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:21.803 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:21.801+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:21.816 INFO:teuthology.orchestra.run.smithi177.stdout:5983 read 0x816f4a6 thru 0x816fb24 (0x67f bytes) 2024-05-08T03:21:21.816 INFO:teuthology.orchestra.run.smithi177.stdout:5984 write 0x149629f thru 0x149ab01 (0x4863 bytes) 2024-05-08T03:21:21.822 INFO:teuthology.orchestra.run.smithi177.stdout:5985 write 0x438ad68 thru 0x438dc3e (0x2ed7 bytes) 2024-05-08T03:21:21.830 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:21.830 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:21.834 INFO:teuthology.orchestra.run.smithi177.stdout:5987 write 0x90001ce thru 0x900da13 (0xd846 bytes) 2024-05-08T03:21:21.853 INFO:teuthology.orchestra.run.smithi177.stdout:5988 writesame 0xd74b12d thru 0xd759820 data_size 0x34(0xe6f4 bytes) 2024-05-08T03:21:21.853 INFO:teuthology.orchestra.run.smithi177.stdout:5989 write 0xd18a687 thru 0xd18e3b7 (0x3d31 bytes) 2024-05-08T03:21:21.856 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:21.857 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:21.860 INFO:teuthology.orchestra.run.smithi177.stdout:5990 trunc from 0xd759821 to 0xb2c03cc 2024-05-08T03:21:21.863 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:21.863 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:21.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:21.877 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:21.880 INFO:teuthology.orchestra.run.smithi177.stdout:5991 write 0xc344d43 thru 0xc353d4e (0xf00c bytes) 2024-05-08T03:21:21.892 INFO:teuthology.orchestra.run.smithi177.stdout:5992 writesame 0x27e0ef4 thru 0x27e73ce data_size 0x1(0x64db bytes) 2024-05-08T03:21:21.898 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:21.898 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:21.909 INFO:teuthology.orchestra.run.smithi177.stdout:5994 write 0xd8d77ee thru 0xd8dcb0b (0x531e bytes) 2024-05-08T03:21:21.916 INFO:teuthology.orchestra.run.smithi177.stdout:5997 read 0x153b671 thru 0x153ffb3 (0x4943 bytes) 2024-05-08T03:21:21.916 INFO:teuthology.orchestra.run.smithi177.stdout:5999 read 0x870dce3 thru 0x871a7e2 (0xcb00 bytes) 2024-05-08T03:21:21.917 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:21.915+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:21.919 INFO:teuthology.orchestra.run.smithi177.stdout:6000 trunc from 0xd8dcb0c to 0xb821f70 2024-05-08T03:21:21.919 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:21.920 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:21.938 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:21:21.941 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:21.941 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:21.959 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:21.960 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:21.960 INFO:teuthology.orchestra.run.smithi177.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-05-08T03:21:21.978 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:21.978 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:21.991 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:21.991 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:21.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:21.991 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:21.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.11523464, 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:21.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:21.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:21.998 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773293448, 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:21.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.002 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773278049, 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:22.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:22.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:22.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:22.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773246407, 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.77323307499999, 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.011 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:22.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:22.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773223281, 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.015 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.019 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773213868, 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:22.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:22.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:22.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773204347, 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:22.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:22.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:22.027 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.77319500599999, 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.028 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:22.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:22.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:22.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:22.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773185791, 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:22.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:22.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:22.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773176429, 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.037 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773166997, 2024-05-08T03:21:22.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.045 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.77315758100001, 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.77314856300001, 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:22.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:22.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:22.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:22.053 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.77313952, 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773127227, 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.058 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.062 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.77311795999999, 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773104102, 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:22.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:22.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 108.773094809, 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.071 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.073 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:22.077 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.780080676, 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:22.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:22.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:22.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:22.081 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:22.082 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:22.083 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:22.083 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:22.081+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:22.184 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:22.182+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:22.228 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:22.229 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:22.229 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:22.229 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:22.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:22.284 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:22.283+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:22.385 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:22.383+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:22.386 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:22.386 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:22.386 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:22.386 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:22.387 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:22.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:22.485 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:22.483+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:22.540 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:22.540 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:22.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:22.540 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:22.540 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:22.548 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:22.593 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:22.592+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:22.656 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:22.656 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:22.656 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:22.656 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:22.665 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:22.687 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:22.686+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:22.788 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:22.787+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:22.811 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:22.811 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:22.812 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:22.812 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:22.812 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:22.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:22.889 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:22.888+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:22.937 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:21:22.976 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:22.976 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:22.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.77425500299999, 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.977 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:22.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:22.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.980 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.774245553, 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:22.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:22.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:22.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:22.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.774232817, 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:22.985 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:22.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.774222241, 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:22.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:22.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:22.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:22.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.774212854, 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:22.994 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:22.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:22.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.77420367400001, 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:22.998 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:22.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:23.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543378639, 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:23.002 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543346402, 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.010 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543333072, 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.011 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543318098, 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.015 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:23.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:23.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543306789, 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.019 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:23.023 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.54328912, 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:23.027 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543273922, 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:23.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.031 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 114.543259828, 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:23.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:23.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:23.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.035 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 106.76830585499999, 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:23.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:23.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 105.845412053, 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 105.845398565, 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 105.845369584, 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 105.84534297800001, 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:23.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 105.412594717, 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.057 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.062 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:23.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:23.064 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:23.062+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:23.164 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:23.163+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:23.209 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:23.209 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:23.210 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:23.210 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:23.219 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:23.265 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:23.263+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:23.364 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:23.364 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:23.364 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:23.364 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:23.364 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:23.365 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:23.364+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:23.373 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:23.466 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:23.464+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:23.521 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:23.521 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:23.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:23.521 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:23.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.521 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 93.586422572000004, 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:23.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.529 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.726246042, 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:23.534 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.726220017000003, 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:23.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:23.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:23.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.539 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.726178001999997, 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.725195325999998, 2024-05-08T03:21:23.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:23.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:23.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:23.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.549 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.725159410000003, 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.550 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.551 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.551 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.551 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:23.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.551 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.552 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.725134367999999, 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.556 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.558 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:23.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.722724448000001, 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.561 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:23.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.722700154000002, 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.568 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:23.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:23.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:23.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:23.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:23.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.574 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.574 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.574 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:23.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:23.574 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.574 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.722627037000002, 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.578 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.580 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.722598804, 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:23.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:23.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:23.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:23.585 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.722568056999997, 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.586 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:23.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:23.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.591 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.722530698, 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.596 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720928966999999, 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:23.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720874946000002, 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.602 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:23.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.607 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720842240000003, 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:23.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.612 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720817848000003, 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.613 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:23.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:23.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.620 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720798426999998, 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.621 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:23.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.628 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720787147999999, 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:23.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 32.720767453999997, 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:23.633 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:23.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:23.638 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:23.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:23.640 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:23.638+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:23.740 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:23.739+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:23.744 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:23.744 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:23.744 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:23.744 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:23.753 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:23.841 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:23.840+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:23.898 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:23.899 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:23.899 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:23.899 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:23.899 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:23.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:23.942 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:23.940+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:23.991 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.924963583999997, 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.923243114000002, 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.923209673000002, 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.921783934000004, 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:24.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.921691319999994, 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.048 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.052 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.053 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.053 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:24.053 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.921180645999996, 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:24.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.058 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.921153137999994, 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:24.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:24.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:24.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.920746707000006, 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:24.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.920291328999994, 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.068 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:24.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:24.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:24.072 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.919763082000003, 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.919290583999995, 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:24.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:24.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:24.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.081 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.919212451999996, 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.919137030000002, 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.086 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:24.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:24.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:24.090 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.919059867000001, 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.091 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:24.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.095 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.918982344, 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:24.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:24.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.918955432000004, 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:24.100 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.918934222000004, 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.105 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:24.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:24.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.109 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.918914442000002, 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.918359916, 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.114 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:24.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:24.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:24.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 75.918340904000004, 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:24.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:24.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:24.123 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:24.125 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:24.125 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:24.125 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:24.126 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:24.123+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:24.136 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:24.136 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:24.153 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:24.153 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:24.170 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:24.170 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:24.188 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:24.188 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:24.206 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:24.206 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:24.223 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:24.224 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:24.225 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:24.223+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:24.241 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:24.242 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:24.245 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:24.245 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:24.254 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:24.338 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:24.337+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:24.400 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:24.401 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:24.409 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:24.451 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:24.449+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:24.528 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:24.526+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:24.554 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:24.554 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:24.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:24.629 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:24.628+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:24.670 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:24.670 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:24.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:24.729 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:24.728+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:24.785 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:24.785 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:24.793 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:24.837 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:24.835+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:24.898 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:24.898 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:24.906 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:24.931 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:24.930+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:24.940 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:24.940 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:24.940 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-05-08T03:21:24.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-05-08T03:21:24.955 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:21:25.032 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:25.030+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:25.065 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.066 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:25.095 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-05-08T03:21:25.138 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:25.137+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:25.222 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.222 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.230 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:25.233 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:25.231+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:25.255 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-05-08T03:21:25.334 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:25.332+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:25.335 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.335 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:25.434 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-05-08T03:21:25.435 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:25.434+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:25.490 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.490 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:25.536 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:25.534+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:25.637 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:25.635+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:25.644 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.644 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:25.739 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:25.737+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:25.800 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.800 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.809 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:25.850 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:25.848+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:25.939 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:25.937+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:25.956 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:25.956 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:25.963 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:21:25.966 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:26.040 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:26.039+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:26.116 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:26.116 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:26.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:26.142 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:26.141+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:26.231 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:26.231 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:26.240 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:26.243 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:26.241+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:26.254 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:26.344 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:26.342+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:26.348 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:26.348 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:26.358 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:26.407 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:26.407 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:26.425 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:26.425 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:26.445 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:26.443+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:26.458 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:26.458 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:26.475 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:26.476 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:26.492 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:26.493 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:26.510 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:26.510 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:26.516 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:26.516 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:26.516 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:26.516 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:26.516 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:26.526 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:26.528 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:26.529 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:26.546 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:26.544+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:26.546 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:26.546 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.636 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.43429252999999, 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.43428308, 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.641 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:26.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:26.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:26.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.434270344, 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.434259768, 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.434250381, 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:26.654 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.658 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.43424120100001, 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:26.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:26.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.662 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203416166, 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.663 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:26.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:26.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203383929, 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.667 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:26.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:26.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203370599, 2024-05-08T03:21:26.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:26.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.675 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203355625, 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.676 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:26.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.680 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203344316, 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:26.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:26.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:26.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203326647, 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:26.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:26.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203311449, 2024-05-08T03:21:26.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:26.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:26.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 118.203297355, 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.694 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:26.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:26.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 110.42834338199999, 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.698 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:26.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.50544958, 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:26.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.505436092, 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.708 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:26.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:26.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.505407111, 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:26.712 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:26.716 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.50538050500001, 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:26.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.072632244, 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:26.721 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:26.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:26.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.726 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.727 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:26.727 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:26.727 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:26.728 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:26.729 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:26.727+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:26.828 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:26.827+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:26.875 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:26.875 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:26.876 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:26.876 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:26.884 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:26.930 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:26.928+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:27.030 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:27.028+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:27.045 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:27.045 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:27.045 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:27.045 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:27.045 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:27.055 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:27.139 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:27.138+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:27.203 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:27.203 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:27.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:27.203 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:27.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 97.268345980000007, 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.204 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:27.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:27.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:27.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.408169450000003, 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:27.211 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:27.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:27.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:27.216 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.408143424999999, 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:27.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:27.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.40810141, 2024-05-08T03:21:27.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:27.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:27.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:27.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:27.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:27.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:27.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:27.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:27.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:27.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.230 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.407118734000001, 2024-05-08T03:21:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.231 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:27.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:27.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.236 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:27.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.242 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.242 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:27.242 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.407082817999999, 2024-05-08T03:21:27.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:27.242 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.242 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.243 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:27.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:27.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:27.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:27.248 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:27.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.248 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.248 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.248 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:27.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:27.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:27.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:27.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:27.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.407057776000002, 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:27.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:27.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.257 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.259 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.404647855999997, 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:27.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:27.265 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.404623561999998, 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.266 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:27.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:27.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.404550444999998, 2024-05-08T03:21:27.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.275 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.404522212000003, 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.277 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:27.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:27.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:27.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.404491465, 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.282 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:27.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:27.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:27.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.286 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.404454106000003, 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.290 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:27.291 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402852375000002, 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:27.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402798353999998, 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.296 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:27.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:27.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:27.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:27.300 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402765647999999, 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:27.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:27.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:27.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:27.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.305 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402741255999999, 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:27.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:27.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:27.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402721835000001, 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:27.310 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402710556000002, 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.315 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:27.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:27.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:27.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:27.319 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 36.402690862, 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.320 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.324 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.325 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.325 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.325 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.325 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:27.326 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:27.327 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:27.324+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:27.427 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:27.426+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:27.473 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:27.473 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:27.474 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:27.474 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:27.482 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:27.566 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:27.565+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:27.627 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:27.628 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:27.628 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:27.628 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:27.628 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:27.629 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:27.627+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:27.636 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:27.730 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:27.728+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:27.783 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:27.783 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:27.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:27.783 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:27.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.783 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.681238425999993, 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.679517955999998, 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:27.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.679484514999999, 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:27.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:27.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.678058776, 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:27.797 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.798 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:27.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:27.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.677966162000004, 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.802 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.806 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.677455488000007, 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:27.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:27.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:27.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.677427980000004, 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:27.811 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:27.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.677021549000003, 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.816 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:27.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:27.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:27.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.676566171000005, 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:27.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.676037923999999, 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.825 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:27.826 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:27.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:27.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.829 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675565426000006, 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.830 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.831 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.832 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.833 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.834 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675487294000007, 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:27.835 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:27.836 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:27.837 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:27.838 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675411871999998, 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.839 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.840 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:27.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675334708999998, 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675257185999996, 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.848 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:27.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:27.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675230274, 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.853 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:27.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:27.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675209064000001, 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.857 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.861 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.675189283999998, 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.862 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:27.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:27.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:27.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.674634757999996, 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:27.866 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.870 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 79.674615746000001, 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.871 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:27.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:27.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:27.875 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:27.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:27.877 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:27.875+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:27.977 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:27.975+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:27.980 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:21:28.037 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:28.037 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:28.037 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:28.049 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:28.078 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:28.076+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:28.165 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:28.165 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:28.165 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:28.165 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:28.165 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:28.174 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:28.217 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:28.216+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:28.280 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:28.278+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 89.284238228999996, 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:21:28.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 89.284170764999999, 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.290 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.291 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:21:28.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:21:28.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.294 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.295 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:21:28.296 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.297 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 89.224336433000005, 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.298 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:21:28.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:21:28.300 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.301 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:21:28.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 88.157828420000001, 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:21:28.303 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.304 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.305 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:21:28.306 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:21:28.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:21:28.308 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 88.131290774000007, 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.309 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:28.310 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.311 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:21:28.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.313 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.313 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:21:28.313 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:21:28.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.313 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:28.314 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:21:28.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:21:28.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.315 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.315 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.315 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:21:28.315 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.316 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:21:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 88.103379005999997, 2024-05-08T03:21:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:21:28.318 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.319 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.319 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.319 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.319 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.319 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.319 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.321 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:21:28.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.323 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:21:28.324 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:21:28.324 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.324 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.324 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.324 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:21:28.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:28.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.325 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.325 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:28.325 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:21:28.325 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.326 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.327 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.327 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.327 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.327 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.327 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:28.327 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.024798812999997, 2024-05-08T03:21:28.328 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:21:28.328 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.328 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.328 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.328 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.329 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.329 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:21:28.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.329 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.329 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.329 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:21:28.331 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.332 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.332 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:21:28.332 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:21:28.332 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.332 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.333 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:21:28.333 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:21:28.333 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.333 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.333 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:21:28.334 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:21:28.334 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.334 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.334 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.334 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:21:28.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:21:28.335 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.335 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.335 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.335 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:21:28.335 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.336 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.337 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.337 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.337 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.337 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.337 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:28.337 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020171437000002, 2024-05-08T03:21:28.338 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:21:28.338 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.338 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.338 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.338 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.339 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.339 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:21:28.339 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.339 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.339 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.339 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:28.340 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.340 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.340 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.340 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.340 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:28.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.341 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.341 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.341 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:28.341 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.342 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.343 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:21:28.344 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.345 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.346 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020150802000003, 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.347 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.348 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:21:28.349 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:21:28.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.351 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020128550999999, 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:28.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020108120000003, 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.356 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:28.360 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020087894, 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:21:28.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:21:28.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020067099999999, 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.367 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.368 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 60.020041085999999, 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.369 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:21:28.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:21:28.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.840084165999997, 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.375 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.840065062000001, 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.383 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.839503487999998, 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:21:28.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:28.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.834289626999997, 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:21:28.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.393 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 59.834122286000003, 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:28.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.471368689000002, 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:28.399 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.403 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.404 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.404 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:28.404 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:28.404 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:28.405 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:28.406 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:28.403+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:28.506 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:28.505+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:28.513 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:28.513 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:28.513 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:28.513 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:28.522 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:28.558 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:28.607 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:28.605+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:28.626 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:28.626 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:28.626 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:28.627 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:28.627 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:28.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:28.682 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:28.682 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:28.699 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:28.699 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:28.708 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:28.706+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:28.716 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:28.716 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:28.732 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:28.733 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:28.743 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.743 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.967767354000003, 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.744 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.745 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:28.747 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:28.748 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.749 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.750 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:28.751 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:28.752 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.753 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:28.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.953144149000003, 2024-05-08T03:21:28.756 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.757 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:28.758 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.759 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.760 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.761 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.762 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.764 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.765 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.766 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:28.767 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:28.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.769 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.770 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:28.771 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.952334870000001, 2024-05-08T03:21:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.772 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.773 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.774 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.775 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.776 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:28.777 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.778 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:28.779 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:28.780 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:28.781 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.782 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.951395593000001, 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.783 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.784 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.786 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.787 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.788 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.949762522999997, 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.789 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.790 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.791 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:28.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:28.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:28.794 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.948324026999998, 2024-05-08T03:21:28.795 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.796 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.797 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:28.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:28.799 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.948304139000001, 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.800 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.801 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.802 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:28.803 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:28.804 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.948280220000001, 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:28.805 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:28.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:28.808 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:28.809 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.94730904, 2024-05-08T03:21:28.810 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:28.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:28.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:28.814 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.947297431000003, 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:28.815 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.816 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:28.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:28.818 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.819 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:28.820 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.947287713000001, 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:28.821 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.822 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.823 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.824 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.825 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.947277251000003, 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.826 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.828 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.829 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.830 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.944700304999998, 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:28.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:28.833 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.834 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:28.835 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.944670058, 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:28.836 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:28.837 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.838 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.839 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.840 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.944643040999999, 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:28.841 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.842 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.843 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.844 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.944624048000001, 2024-05-08T03:21:28.845 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:28.846 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:28.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:28.848 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:28.849 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.944593988999998, 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:28.850 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:28.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.942809525000001, 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.855 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:28.856 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.942797702, 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:28.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.864 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 37.942785845000003, 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:28.865 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:28.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:28.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:28.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:28.869 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.870 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.870 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.870 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:28.870 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:28.870 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:28.871 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:28.871 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:28.872 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:28.872 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:28.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:28.873 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:28.873 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:28.873 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:28.874 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:28.875 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:28.870+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:28.972 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:28.970+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:28.976 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:21:28.983 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:28.983 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:28.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:29.072 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:29.071+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:29.108 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.108 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.118 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:29.174 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:29.173+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:29.223 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.223 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.232 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:29.274 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:29.273+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:29.337 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.337 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:29.390 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:29.388+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:29.451 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.452 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:29.501 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:29.499+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:29.577 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:29.575+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:29.605 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.605 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.614 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:29.698 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:29.696+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:29.759 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.759 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:29.778 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:29.777+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:29.879 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:29.878+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:29.929 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:29.929 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:29.939 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:29.981 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:29.978+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:30.082 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:30.080+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:30.085 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.085 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:30.183 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:30.181+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:30.208 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.208 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.218 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:30.283 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:30.282+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:30.322 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.322 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.331 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:30.415 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:30.414+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:30.477 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.477 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.486 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:30.487 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:30.483+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:30.572 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:30.572 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:30.572 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-05-08T03:21:30.573 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T03:21:30.573 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T03:21:30.573 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-05-08T03:21:30.617 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:30.615+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:30.631 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.632 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.641 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:30.680 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:30.681 INFO:teuthology.orchestra.run.smithi188.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T03:21:30.681 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:30.687 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:30.685+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:30.747 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.748 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.756 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:30.800 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:30.799+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:30.869 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:30.869 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:30.876 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:30.879 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:30.888 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:30.886+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:30.989 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:30.987+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:30.992 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:21:31.003 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:31.003 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:31.021 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:31.022 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:31.031 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:31.040 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:31.040 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:31.042 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:31.057 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:31.057 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:31.074 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:31.075 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:31.090 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:31.088+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:31.093 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:31.126 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:31.158 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:31.159 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:31.192 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:31.192 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:31.193 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:31.191+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:31.201 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:31.292 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:31.290+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:31.349 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.246456957000007, 2024-05-08T03:21:31.350 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.351 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.352 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.353 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.354 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.355 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:31.356 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.244736486999997, 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.357 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.358 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.359 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.360 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:31.361 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.244703045999998, 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.362 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.363 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.364 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.365 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.366 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.243277307, 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:31.368 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.369 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:31.370 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.371 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.243184693000003, 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.372 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.373 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.374 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.375 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:31.376 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.242674019000006, 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:31.377 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:31.378 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.379 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.380 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:31.381 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.382 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.242646511000004, 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.383 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:31.384 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.385 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:31.386 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.242240080000002, 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.387 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.388 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:31.389 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.390 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.391 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:31.392 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.241784702000004, 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:31.393 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.394 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.395 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:31.396 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.241256454999998, 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.397 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.398 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.399 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.400 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.401 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240783957000005, 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:31.402 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.403 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.404 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.405 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240705825000006, 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.407 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:31.408 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:31.409 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.410 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.411 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240630402999997, 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.412 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.413 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.414 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.415 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240553239999997, 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.416 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.417 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.418 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:31.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.420 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240475716999995, 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240448805, 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.425 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:31.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.429 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.429 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:31.429 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:31.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.429 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.430 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.430 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:31.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:31.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:31.431 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240427595, 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.432 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:31.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.240407814999998, 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.438 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:31.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:31.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.442 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.239853288999996, 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 83.239834277, 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.447 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:31.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:31.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.452 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:31.453 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:31.454 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:31.452+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:31.554 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:31.553+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:31.599 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:31.599 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:31.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:31.600 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:31.609 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:31.656 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:31.654+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:31.712 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:31.712 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:31.712 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:31.712 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:31.712 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:31.721 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:31.757 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:31.755+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:31.782 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:21:31.839 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:31.839 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 92.840553557999996, 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.840 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.841 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:21:31.842 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:21:31.843 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:21:31.844 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 92.840486093999999, 2024-05-08T03:21:31.845 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.846 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:21:31.847 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.848 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:31.850 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:31.851 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:31.852 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 92.780651762000005, 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.853 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.854 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.855 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:21:31.856 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:21:31.857 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.714143749000002, 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:21:31.858 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.859 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.860 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.861 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:31.862 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.863 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.687606102999993, 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.864 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.865 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.866 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.867 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.868 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.659694334999998, 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.581114141999997, 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.873 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.874 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.875 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.876 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:31.877 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576486766000002, 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:31.878 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:21:31.879 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.880 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:31.881 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576466130999997, 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.882 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.883 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.884 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576443879999999, 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:21:31.885 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.886 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.887 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:21:31.888 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576423449000004, 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.889 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.890 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.891 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.892 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576403223, 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.893 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:21:31.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.896 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576382428999999, 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.897 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.898 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.899 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.900 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:31.901 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.576356414999999, 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:21:31.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:21:31.904 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.905 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.396399494999997, 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.906 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:21:31.907 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:21:31.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.909 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.396380391000001, 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.910 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:21:31.911 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.912 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:31.913 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.395818816999999, 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.390604955999997, 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.918 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:31.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 63.390437615000003, 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.923 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:31.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:21:31.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:21:31.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:21:31.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.929 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 55.027684018000002, 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:21:31.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:21:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:21:31.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.934 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.934 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:31.934 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:31.934 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:31.935 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:31.936 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:31.933+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:32.036 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:32.035+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:32.042 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:32.042 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:32.042 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:32.042 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:32.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:32.137 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:32.135+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:32.197 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:32.197 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:32.197 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:32.197 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:32.197 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:32.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:32.238 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:32.236+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:32.339 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:32.337+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.57847245, 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.354 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.355 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:32.356 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:32.357 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.358 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.359 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.563849245, 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.360 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:32.361 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:32.362 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:32.363 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:32.364 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:32.365 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.563039965999998, 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.366 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.367 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.368 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.369 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:32.370 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.371 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.372 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.562100688999998, 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.373 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:32.374 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:32.375 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.377 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.378 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.560467619000001, 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:32.379 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.380 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:32.381 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.382 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.383 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.384 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.385 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.559029123000002, 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.386 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.387 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.388 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.389 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:32.390 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.559009234999998, 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.391 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.392 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.393 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.394 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.395 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.558985315999998, 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.396 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.397 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.398 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.399 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:32.400 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.558014135999997, 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:32.401 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.402 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.403 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.404 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.558002526999999, 2024-05-08T03:21:32.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:32.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.557992808999998, 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:32.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:32.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:32.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.557982346999999, 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.415 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:32.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:32.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.555405401000002, 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.555375154000004, 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:32.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.555348137000003, 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.430 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:32.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.434 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.555329143999998, 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.435 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:32.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:32.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:32.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.555299085000001, 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.440 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.444 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.553514620999998, 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:32.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:32.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:32.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.553502797999997, 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.449 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:32.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.453 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 41.553490941, 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.454 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:32.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:32.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:32.458 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.459 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.459 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:32.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:32.461 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:32.458+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:32.561 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:32.560+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:32.566 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:32.567 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:32.567 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:32.567 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:32.575 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:32.663 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:32.661+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:32.681 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:32.681 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:32.681 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:32.681 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:32.681 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:32.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:32.774 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:32.772+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 116.63901148799999, 2024-05-08T03:21:32.850 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:32.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:32.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:32.854 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.855 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.719281177, 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:32.856 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:32.857 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.858 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.859 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.71924946, 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.860 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:32.861 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:32.862 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.863 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.719224356, 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.864 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.865 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:32.866 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:32.867 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.719199551, 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.868 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.869 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.870 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.871 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.719171007, 2024-05-08T03:21:32.872 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.873 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.874 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:32.875 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.876 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.71914848500001, 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.877 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:32.878 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:32.879 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.880 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.719118145, 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.881 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.882 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:32.883 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:32.884 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 115.719096964, 2024-05-08T03:21:32.885 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.886 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.887 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.888 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 101.906455028, 2024-05-08T03:21:32.889 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.890 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:32.891 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:32.892 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:32.893 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:32.894 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.895 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.717780306999998, 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.896 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.897 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.898 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.899 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.900 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.695576963999997, 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.901 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.902 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.903 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.904 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.693880845999999, 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.905 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.906 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:32.907 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:32.908 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.674176058, 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:32.909 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.910 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:32.911 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.912 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.913 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.914 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.674152097999993, 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:32.915 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.916 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.917 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:32.918 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:32.919 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.920 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.633560552999995, 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.921 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.922 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.923 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.924 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 92.620786656000007, 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.925 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.926 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.927 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:32.928 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 73.360047722000004, 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.929 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.930 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.931 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.932 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.001743322999999, 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.933 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:21:32.934 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:21:32.935 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:21:32.936 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 18.000124585999998, 2024-05-08T03:21:32.937 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.938 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.939 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.940 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:32.941 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:32.943 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:32.943 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:32.942+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:33.044 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:33.042+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:33.089 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:33.089 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:33.089 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:33.089 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:33.097 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:33.174 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:33.182 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:33.180+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:33.243 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:33.244 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:33.244 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:33.244 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:33.244 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:33.245 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:33.243+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:33.252 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:33.299 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:33.299 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:33.316 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:33.316 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:33.332 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:33.333 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:33.346 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:33.344+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:33.349 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:33.349 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:33.366 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:33.366 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:33.384 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:33.384 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:33.402 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:33.403 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 121.52741839399999, 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:33.404 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:33.405 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.406 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.407 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:33.408 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:33.409 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.410 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185477202, 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.411 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:33.412 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:33.413 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:33.414 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185461803, 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.415 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.416 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:33.417 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.418 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.419 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185430161, 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:33.420 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:33.421 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.422 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.423 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185416829, 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.424 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.425 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:33.426 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185407035, 2024-05-08T03:21:33.428 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.429 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:33.430 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.431 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.432 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185397622, 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.433 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.434 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.435 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:33.436 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185388101, 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.437 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.438 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:33.439 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:33.440 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.441 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.18537876000001, 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:33.442 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:33.443 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.444 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.445 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185369545, 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.446 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.447 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.448 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:33.449 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185360183, 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.450 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.451 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:33.452 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:33.453 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185350751, 2024-05-08T03:21:33.454 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:33.455 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.456 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185341335, 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185332317, 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185323274, 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:33.469 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.470 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.471 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185310981, 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.472 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.473 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.474 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.475 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.476 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185301714, 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.477 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.478 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.479 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:33.480 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185287856, 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.481 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:33.483 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.484 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:33.485 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.185278563, 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:33.486 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.487 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.488 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.489 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 117.19226442999999, 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:33.494 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:33.495 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:33.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:33.497 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:33.494+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:33.497 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:33.596 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:33.594+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:33.642 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:33.642 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:33.651 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:33.697 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:33.695+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:33.798 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:33.796+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:33.798 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:33.798 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:33.804 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:21:33.808 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:33.899 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:33.897+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:33.926 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:33.926 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:33.936 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:34.000 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:33.998+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:34.045 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.045 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.055 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:34.100 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:34.099+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:34.160 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.160 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.168 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:34.210 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:34.208+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:34.303 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:34.301+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:34.313 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.313 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.323 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:34.403 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:34.402+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:34.428 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.428 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.437 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:34.521 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:34.520+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:34.583 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.584 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.593 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:34.635 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:34.634+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:34.706 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:34.705+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:34.740 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.740 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.749 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:34.777 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:21:34.838 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:34.837+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:34.897 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:34.897 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:34.906 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:34.947 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:34.947+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:35.009 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:35.008+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:35.051 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:35.051 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:35.060 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:35.144 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:35.144+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:35.211 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:35.210+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:35.216 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:35.313 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:35.311+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:35.360 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:35.360 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:35.370 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:35.413 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:35.412+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:35.475 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:35.475 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:35.484 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:35.498 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:35.527 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:35.526+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:35.591 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:35.591 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:35.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:35.614 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:35.614+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:35.650 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:35.650 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:35.667 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:35.667 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:35.684 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:35.684 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:35.689 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:35.689 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:35.689 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T03:21:35.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:21:35.701 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:35.701 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:35.715 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:35.715+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:35.719 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:35.720 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:35.720 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:35.720 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:35.720 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:35.729 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:35.737 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:35.737 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:35.754 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:35.754 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:35.772 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:35.772 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:35.825 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:35.825+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:35.845 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:35.845 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:35.845 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:35.845 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:35.845 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:35.856 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:35.917 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:35.915+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.191354613999998, 2024-05-08T03:21:35.967 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.968 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.969 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.970 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:35.971 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.972 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.176731408999999, 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.973 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:35.974 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:35.975 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:35.976 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:35.977 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:35.978 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.175922130000004, 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:35.979 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.980 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.981 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:35.982 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.983 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.984 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.174982853000003, 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:35.985 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:35.986 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:35.987 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.988 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:35.989 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:35.990 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.173349782999999, 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:35.991 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.992 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.993 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.994 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.995 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.996 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.171911287, 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:35.997 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:35.998 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:35.999 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:36.000 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:36.001 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.171891399000003, 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.002 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.003 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:36.004 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:36.005 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.006 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.171867480000003, 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.007 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.008 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.009 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.010 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.011 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.170896300000003, 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.013 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.170884690999998, 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.016 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:36.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:36.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.020 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.170874972999997, 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.021 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:36.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:36.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.170864510999998, 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:36.030 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.168287565, 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:36.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:36.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.168257318000002, 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.035 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.168230301000001, 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:36.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:36.044 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.168211307999997, 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.045 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.168181249, 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:36.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.166396785000003, 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:36.054 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.166384962000002, 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:36.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:36.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:36.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:36.063 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 45.166373104999998, 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:36.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:36.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:36.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:36.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.068 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.069 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:36.071 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:36.071 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 135, 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:21:35.805888+0000", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:21:03.827528+0000", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:20:46.806790+0000", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5806080, 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "nodeep-scrub", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "noscrub", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:21:36.072 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 6, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "luminous", 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:21:36.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:21:36.074 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:21:36.075 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:21:36.076 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:21:36.077 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 3, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 3, 2024-05-08T03:21:36.078 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.5, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.5, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.5, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.83005142211914062, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 0.82999998331069946, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:21:36.079 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:21:36.080 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:21:36.081 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:21:36.082 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:21:36.083 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 8, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 8, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.25, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:21:36.084 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.83005142211914062, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 0.82999998331069946, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:21:36.085 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:21:36.086 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "135", 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 49, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 135, 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:21:36.087 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:21:36.088 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:21:36.089 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 3.7599999904632568, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 3.7599999904632568, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.5, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.83005142211914062, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 0.82999998331069946, 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.090 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 40, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.091 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.093 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:21:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.096 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 120, 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.098 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.100 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:36.101 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 120, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.104 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.106 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 98, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.107 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.108 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:21:36.109 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.111 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0.5313873291015625, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 99, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.112 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.113 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.115 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.116 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 14, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 58, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 106, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 45, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 61, 2024-05-08T03:21:36.117 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.119 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1469026711", 2024-05-08T03:21:36.121 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1469026711", 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1469026711", 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1469026711", 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0.1090240478515625, 2024-05-08T03:21:36.122 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 120, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.124 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.125 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:36.127 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.129 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.130 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:36.131 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.132 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:36.133 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:21:36.134 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:36.135 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 61 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:21:36.136 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:36.137 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [ 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "1.5", 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: 2, 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: 0 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.19", 2024-05-08T03:21:36.138 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: 4 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [ 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.6", 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 1, 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:36.140 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [ 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "snaps": [ 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "begin": 36, 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "length": 5 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [ 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "snaps": [ 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:36.141 INFO:teuthology.orchestra.run.smithi177.stdout: "begin": 36, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "length": 1 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:21:36.142 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:36.143 INFO:tasks.thrashosds.thrasher:j is {'epoch': 135, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:21:35.805888+0000', 'last_up_change': '2024-05-08T03:21:03.827528+0000', 'last_in_change': '2024-05-08T03:20:46.806790+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8300514221191406, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300514221191406, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 49, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.759999990463257, 'score_stable': 3.759999990463257, 'optimal_score': 0.5, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8300514221191406, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5313873291015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 58, 'up_from': 106, 'up_thru': 45, 'down_at': 61, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1469026711}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1469026711}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1469026711}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1469026711}]}, 'public_addr': '172.21.15.188:6817/1469026711', 'cluster_addr': '172.21.15.188:6819/1469026711', 'heartbeat_back_addr': '172.21.15.188:6823/1469026711', 'heartbeat_front_addr': '172.21.15.188:6821/1469026711', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1090240478515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 61}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.5', 'osds': [2, 0]}, {'pgid': '3.19', 'osds': [3, 4]}], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:21:36.144 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 1.5 2024-05-08T03:21:36.144 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 1.5 2024-05-08T03:21:36.144 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:36.069+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:36.189 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:36.187+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:36.252 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:36.252 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:36.252 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:36.252 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:36.260 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:36.302 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:36.300+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:36.372 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:36.370+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:36.407 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:36.407 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:36.407 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:36.407 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:36.407 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:36.415 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:36.500 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:36.498+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:36.562 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:36.562 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:36.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:36.562 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:36.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.562 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.35151370299999, 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:36.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:36.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.431783392, 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:36.568 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:36.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:36.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.572 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.431751675, 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:36.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.431726571, 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.578 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:36.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:36.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:36.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.431701766, 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:36.582 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.586 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.431673222, 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.43165070000001, 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:36.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.43162036, 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:36.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:36.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 119.431599179, 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 105.618957243, 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.603 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:36.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:36.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:36.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:36.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:36.609 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.430282521999999, 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.610 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:36.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:36.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:36.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:36.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.408079178999998, 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.615 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.619 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.406383061, 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.386678273000001, 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:36.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:36.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.386654312999994, 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:36.630 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:36.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:36.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:36.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.637 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.346062767999996, 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:36.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:36.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 96.333288870999993, 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.642 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:36.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.072549937000005, 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:36.646 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:36.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:36.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.714245538, 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.654 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 21.712626800999999, 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:36.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:36.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:36.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:36.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:36.659 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:36.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:36.661 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:36.660+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:36.761 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:36.759+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:36.797 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:21:36.807 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:36.807 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:36.808 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:36.816 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:36.824 INFO:teuthology.orchestra.run.smithi177.stderr:clear 1.5 pg_upmap mapping 2024-05-08T03:21:36.901 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:36.899+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:36.962 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:36.962+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:36.969 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:36.969 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:36.969 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:36.969 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:36.969 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:36.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:37.064 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:37.063+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:37.125 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 125.249330559, 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.126 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:37.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:37.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90738936699999, 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:37.132 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907373968, 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:37.136 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.138 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.140 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90734232600001, 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.141 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:37.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907328994, 2024-05-08T03:21:37.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.149 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.9073192, 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:37.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:37.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.153 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907309787, 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90730026599999, 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907290925, 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.165 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90728171000001, 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.166 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.169 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90727234800001, 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.170 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:37.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.173 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90726291599999, 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:37.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:37.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:37.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.177 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.9072535, 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.178 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907244482, 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.182 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.183 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907235439, 2024-05-08T03:21:37.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:37.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90722314600001, 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.191 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.192 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.195 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907213879, 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.196 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.90720002099999, 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.200 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.907190728, 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.204 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:37.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:37.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:37.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.914176595, 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:37.208 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:37.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.211 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:37.212 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:37.214 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:37.214 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:37.213+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:37.315 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:37.313+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:37.320 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:37.320 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:37.320 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:37.320 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:37.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:37.416 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:37.414+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:37.474 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:37.474 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:37.474 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:37.474 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:37.474 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:37.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:37.523 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:37.521+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:37.617 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:37.617+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:37.626 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:37.626 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:37.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:37.627 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:37.627 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:37.635 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:37.718 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:37.717+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:37.746 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:37.746 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:37.746 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:37.754 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:37.784 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:37.819 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:37.818+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:37.832 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:21:37.880 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:37.880 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:37.880 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:37.880 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:37.880 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:37.891 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:37.934 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:37.933+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:37.940 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:37.940 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:37.960 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:37.960 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:37.993 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:37.993 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:38.022 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:38.020+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:38.025 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:38.026 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:38.043 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:38.043 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:38.060 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:38.060 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:38.065 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.065 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.080 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:38.080 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:38.080 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:38.096 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:38.096 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:38.149 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:38.148+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:38.201 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.201 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.210 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:38.252 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:38.251+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:38.324 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:38.323+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:38.355 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.355 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.364 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:38.425 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:38.425+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:38.469 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.469 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.478 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:38.562 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:38.562+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:38.625 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.625 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.627 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:38.626+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:38.634 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:38.728 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:38.727+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:38.780 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.780 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.789 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:38.829 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:38.827+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:38.909 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:38.910 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:38.921 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:38.964 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:38.963+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:39.031 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.031 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.031 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:39.030+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:39.039 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:39.132 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:39.131+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:39.183 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.184 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.192 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:39.235 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:39.233+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:39.297 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.297 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.306 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:39.351 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:39.349+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:39.413 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.414 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:39.436 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:39.435+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:39.537 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:39.536+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:39.567 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.568 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:39.638 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:39.636+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:39.682 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.682 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.691 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:39.775 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:39.774+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:39.839 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:39.837+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:39.844 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:39.844 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:39.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:39.855 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:21:39.940 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:39.940+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:40.002 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:40.002 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:40.011 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:40.052 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:40.051+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:40.108 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:40.143 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:40.141+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:40.156 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:40.156 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:40.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:40.229 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:40.229 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:40.244 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:40.242+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:40.247 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:40.247 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:40.263 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:40.264 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:40.280 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:40.280 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:40.298 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:40.298 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:40.311 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:40.311 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:40.315 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:40.315 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:40.320 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:40.331 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:40.332 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:40.345 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:40.343+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:40.349 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:40.349 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:40.426 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:40.426 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:40.427 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:40.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:40.479 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:40.477+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:40.541 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.665495763, 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.542 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:40.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:40.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.548 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323554571, 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:40.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:40.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323539172, 2024-05-08T03:21:40.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:40.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:40.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:40.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:40.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.558 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32350753, 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:40.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:40.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32349419800001, 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:40.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323484404, 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:40.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323474991, 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.575 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32346547, 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:40.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:40.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32345612899999, 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323446914, 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.584 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:40.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323437552, 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:40.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:40.591 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32342812, 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.592 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:40.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.595 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32341870400001, 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.596 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:40.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:40.599 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32340968600001, 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.600 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:40.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323400643, 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:40.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32338835, 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.608 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.611 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.32337908300001, 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.612 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:40.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.615 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323365225, 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.616 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:40.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:40.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:40.619 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 127.323355932, 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:40.620 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:40.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:40.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 124.330341799, 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:40.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:40.628 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:40.630 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:40.630 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:40.628+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:40.733 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:40.729+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:40.774 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:40.774 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:40.774 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:40.774 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:40.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:40.831 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:40.830+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:40.837 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:21:40.896 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:40.896 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:40.896 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:40.896 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:40.896 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:40.906 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:40.950 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:40.949+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:41.016 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:41.016 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:41.016 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:41.025 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:41.066 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:41.065+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:41.129 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:41.129 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:41.129 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:41.129 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:41.133 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:41.132+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:41.138 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:41.234 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:41.233+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:41.283 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:41.283 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:41.284 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:41.284 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:41.284 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:41.292 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:41.334 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:41.333+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:41.435 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:41.435+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:41.439 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.439 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.237569842, 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:41.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:41.443 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.23756039200001, 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.444 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.237547656, 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.448 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:41.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.452 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.23753708000001, 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:41.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:41.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.237527693, 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:41.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.237518513, 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.461 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:41.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:41.465 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.00669347799999, 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:41.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:41.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.00666124099999, 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:41.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:41.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:41.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.00664791099999, 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:41.474 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.00663293700001, 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.479 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:41.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:41.482 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.006621628, 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.483 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.00660395899999, 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.487 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:41.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.00658876099999, 2024-05-08T03:21:41.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.006574667, 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.499 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 125.231620694, 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.500 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:41.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:41.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:41.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:41.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.308726892, 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.505 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:41.509 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.308713404, 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:41.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:41.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:41.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.513 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.308684423, 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.517 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.308657817, 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.518 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 123.875909556, 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:41.522 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.527 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:41.528 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:41.536 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:41.535+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:41.633 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:41.634 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:41.634 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:41.634 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:41.637 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:41.635+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:41.642 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:41.738 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:41.737+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:41.787 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:41.788 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:41.788 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:41.788 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:41.788 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:41.797 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:41.839 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:41.838+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:41.846 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:41.846 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:41.846 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T03:21:41.846 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T03:21:41.846 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T03:21:41.847 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 111.978915644, 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:41.915 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.916 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.917 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:41.920 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:41.921 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:41.922 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.118739114, 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:41.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:41.925 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:41.926 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.118713089000003, 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.927 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.928 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:41.930 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:41.931 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.118671073999998, 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.932 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:41.933 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:41.934 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:41.935 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:41.936 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.117688397999999, 2024-05-08T03:21:41.937 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:41.938 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:41.939 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:41.940 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:41.941 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.117652481999997, 2024-05-08T03:21:41.942 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.943 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.944 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.945 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.946 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.11762744, 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.947 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.948 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.949 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.950 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.951 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.115217520000002, 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:41.953 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:41.954 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.955 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.956 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.115193226000002, 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.957 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.115120109000003, 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:41.962 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:41.963 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:41.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.115091876000001, 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.967 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:41.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:41.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.970 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.115061128999997, 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.972 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:41.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:41.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:41.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:41.977 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.115023770000001, 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.978 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:41.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:41.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.113422039, 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.113368018000003, 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.992 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.113335311999997, 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:41.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:41.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:41.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:41.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.113310920000004, 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.998 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:41.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:42.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.113291498999999, 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.007 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.11328022, 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.008 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.012 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 51.113260525999998, 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.013 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:42.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:42.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:42.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.017 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:42.019 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:42.020 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:42.020 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T03:21:42.020 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:42.021 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:42.018+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:42.120 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:42.118+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:42.166 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:42.166 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:42.166 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:42.166 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:42.175 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:42.220 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:42.220+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:42.321 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:42.320+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:42.322 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:42.322 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:42.322 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:42.322 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:42.322 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:42.330 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:42.361 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:42.422 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:42.421+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:42.440 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:42.440 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:42.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:42.440 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:42.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.337561179000005, 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.441 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.442 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.443 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.335840708999996, 2024-05-08T03:21:42.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.335807267999996, 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.450 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.452 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.454 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.334381528999998, 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:42.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:42.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:42.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.334288915000002, 2024-05-08T03:21:42.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:42.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:42.463 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.333778241000005, 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.464 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:42.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:42.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.333750733000002, 2024-05-08T03:21:42.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:42.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:42.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:42.472 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.333344302, 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.332888924000002, 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.332360676999997, 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331888179000003, 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:42.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331810047000005, 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:42.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331734624999996, 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:42.495 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.499 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331657461999995, 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:42.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:42.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:42.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331579938999994, 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.504 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:42.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:42.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:42.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331553026999998, 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:42.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:42.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:42.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331531816999998, 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:42.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.331512036999996, 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.330957510999994, 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.522 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 94.330938498999998, 2024-05-08T03:21:42.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:42.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:42.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:42.531 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:42.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:42.533 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:42.533 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:42.533 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:42.533 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:42.534 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:42.534 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:42.534 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:42.530+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:42.547 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:42.547 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:42.566 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:42.566 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:42.584 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:42.584 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:42.602 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:42.602 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:42.620 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:42.620 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:42.632 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:42.632+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:42.656 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:42.656 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:42.665 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:42.749 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:42.748+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:42.811 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:42.811 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:42.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:42.834 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:42.834+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:42.857 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:21:42.935 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:42.933+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:42.980 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:42.980 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:42.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:43.036 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:43.035+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:43.096 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.097 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.105 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:43.137 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:43.135+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:43.237 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:43.237+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:43.249 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.249 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.258 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:43.339 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:43.337+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:43.404 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.404 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:43.440 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:43.439+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:43.519 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.519 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:43.572 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:43.571+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:43.632 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.633 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:43.682 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:43.681+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:43.744 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:43.742+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:43.788 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.788 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.797 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:43.844 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:43.843+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:43.857 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:21:43.911 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:43.912 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:43.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:43.945 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:43.944+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:44.046 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:44.046+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:44.065 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:44.066 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:44.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:44.148 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:44.146+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:44.220 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:44.220 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:44.229 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:44.272 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:44.271+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:44.335 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:44.335 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:44.344 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:44.349 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:44.348+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:44.451 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:44.450+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:44.490 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:44.491 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:44.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:44.584 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:44.583+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:44.632 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:44.644 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:44.645 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:44.653 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:44.678 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:44.676+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:44.753 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:44.752+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:44.759 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:44.759 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:44.777 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:44.777 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:44.793 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:44.794 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:44.794 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:44.794 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:44.802 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:44.802 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:44.804 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:44.821 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:44.821 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:44.841 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:44.841 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:44.854 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:44.853+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:44.871 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:44.871 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:44.890 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:44.890 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:44.912 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:44.912 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:44.942 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:44.942 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:44.942 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:44.943 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:44.943 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:44.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:44.993 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:44.991+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:45.057 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:45.056+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:45.099 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:45.099 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:45.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:45.099 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:45.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.099 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.897094434, 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.103 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.897084984, 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:45.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:45.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.897072248, 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:45.108 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.112 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.89706167200001, 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:45.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:45.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.897052285, 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.120 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.89704310499999, 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.121 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:45.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:45.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:45.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66621807000001, 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.125 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66618583299999, 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.129 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:45.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:45.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66617250300001, 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.133 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.666157529, 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:45.137 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.141 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66614622, 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:45.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:45.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:45.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66612855100001, 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.149 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66611335299999, 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.150 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 136.66609925899999, 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.154 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:45.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 128.89114528600001, 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.158 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:45.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:45.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:45.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:45.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.163 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.96825148400001, 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.167 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.968237996, 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:45.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:45.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.968209015, 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.172 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.173 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.96818240899999, 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.176 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:45.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:45.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.53543414799999, 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:45.180 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:45.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:45.185 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.186 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.186 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.186 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.186 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.186 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:45.187 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:45.188 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:45.186+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:45.288 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:45.286+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:45.335 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:45.335 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:45.335 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:45.335 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:45.344 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:45.389 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:45.388+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:45.488 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:45.489 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:45.489 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:45.489 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:45.489 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:45.490 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:45.489+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:45.497 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:45.591 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:45.590+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.66954147200001, 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:45.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:45.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:45.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:45.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:45.614 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.809364942000002, 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:45.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:45.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:45.619 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.809338916999998, 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.620 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:45.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:21:45.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.809296902, 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:21:45.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:21:45.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:21:45.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:21:45.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.808314226, 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:21:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:21:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.808278309999999, 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:21:45.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:21:45.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:21:45.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.808253268000001, 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.645 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.805843348000003, 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.650 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:21:45.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.805819053999997, 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.655 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.659 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.805745936999998, 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.660 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:21:45.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:21:45.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:21:45.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.805717704000003, 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:21:45.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:21:45.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:21:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.669 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.805686956999999, 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.805649598000002, 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.679 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.804047867000001, 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:45.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.803993845999997, 2024-05-08T03:21:45.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:21:45.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.803961139999998, 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.689 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.803936747999998, 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.694 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.698 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.803917327000001, 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.699 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:21:45.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:21:45.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.803906048000002, 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.704 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 54.803886353999999, 2024-05-08T03:21:45.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:21:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:21:45.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:21:45.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.713 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:45.715 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:45.715 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:45.713+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:45.820 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:45.819+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:45.868 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:45.868 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:45.868 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.868 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(osd.1.0:6201 3.18 e145/97 3:1a5b7e27:::rbd_data.11829e0323ef.0000000000000044:1a v 145'559, mlcod=143'556)", 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:21:45.865869+0000", 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 0.00081061000000000002, 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00083200099999999996, 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865869+0000", 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865869+0000", 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865869+0000", 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3500000000000002e-07 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865875+0000", 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6319999999999996e-06 2024-05-08T03:21:45.870 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865891+0000", 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5650000000000001e-05 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865899+0000", 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.789e-06 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.865996+0000", 2024-05-08T03:21:45.871 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6594000000000004e-05 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:21:45.866016+0000", 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9942999999999999e-05 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 1 2024-05-08T03:21:45.872 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:45.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:21:45.879 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:45.916 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:45.916+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:46.018 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:46.016+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:46.034 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:46.034 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:46.034 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:46.034 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:46.034 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:46.046 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:46.130 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:46.129+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:46.194 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:46.194 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.091885278999996, 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.195 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:46.196 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.197 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.198 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:46.199 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.090164809000001, 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:46.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:46.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:46.203 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.090131368000002, 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:46.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:46.205 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:46.206 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.207 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:46.208 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.088705629000003, 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:46.209 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.210 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.211 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:46.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.088613015000007, 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.213 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.214 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.215 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.216 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.088102340999995, 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:46.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:46.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:46.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:46.221 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.088074832999993, 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:46.222 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.223 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.224 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.225 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:46.226 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.087668402000006, 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.227 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:46.228 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:46.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:46.230 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.087213023999993, 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.231 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:46.232 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:46.233 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.234 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.086684777000002, 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.236 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.237 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:46.238 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.239 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.086212278999994, 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.241 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.242 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.243 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:46.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.086134146999996, 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.245 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.246 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:46.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:46.249 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.250 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:46.251 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.086058725000001, 2024-05-08T03:21:46.252 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.253 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:46.254 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:46.255 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085981562000001, 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.257 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.258 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.259 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.260 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085904038999999, 2024-05-08T03:21:46.261 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.262 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.263 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.264 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:46.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085877127000003, 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.266 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.267 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.268 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.269 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085855917000004, 2024-05-08T03:21:46.270 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.271 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.273 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.274 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085836137000001, 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:46.275 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.276 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.277 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:46.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085281610999999, 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.280 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:46.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:46.282 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:46.283 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 98.085262599000004, 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:46.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:46.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:46.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:46.287 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.288 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:46.290 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:46.290 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:46.288+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:46.390 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:46.390+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:46.436 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:46.437 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:46.437 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:46.437 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:46.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:46.491 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:46.490+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:46.551 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:46.551 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:46.551 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:46.551 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:46.552 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:46.560 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:46.602 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:46.600+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:46.694 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:46.693+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:46.707 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 107.708568888, 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:46.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:46.715 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 107.708501424, 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:46.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:21:46.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:21:46.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.722 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.723 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 107.648667092, 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:21:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:21:46.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:21:46.731 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 106.58215907899999, 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.732 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.733 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.734 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.735 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:21:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.737 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.738 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 106.555621433, 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:21:46.739 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.740 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:21:46.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.742 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:46.743 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:46.744 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 106.527709665, 2024-05-08T03:21:46.745 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:21:46.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:46.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.749 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.449129471999996, 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.750 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.751 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.752 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:21:46.753 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.754 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444502095999994, 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.755 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:46.756 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.757 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:21:46.758 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:21:46.759 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444481460999995, 2024-05-08T03:21:46.760 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.761 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.762 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.762 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.762 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:46.762 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.762 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.763 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.764 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:21:46.765 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.766 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444459210000005, 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:46.768 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.769 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.770 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.771 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444438778999995, 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.772 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.773 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.774 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:21:46.775 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444418553000006, 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.777 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:21:46.778 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:21:46.779 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.780 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444397758999997, 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.781 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.782 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.783 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.784 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.785 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:46.786 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.444371744999998, 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.787 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.788 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:21:46.789 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:21:46.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.791 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.264414825000003, 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.792 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:46.793 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:21:46.794 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.795 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:21:46.796 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.797 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.264395721, 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:21:46.798 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:46.799 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:21:46.800 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.801 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.802 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.263834146999997, 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:46.803 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:46.804 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:21:46.805 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.806 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.258620285999996, 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.809 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:21:46.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:21:46.811 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:21:46.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:46.813 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 78.258452945000002, 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.814 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.815 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.816 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.817 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:21:46.818 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:21:46.819 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.895699347999994, 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:46.820 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.821 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:21:46.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.823 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.825 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:46.826 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:46.826 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:46.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:46.828 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:46.826+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:46.870 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:21:46.924 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:46.954 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:46.954 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:46.954 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:46.954 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:46.964 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:46.968 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:46.967+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:47.021 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:47.021 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:47.021 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:21:47.050 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:47.051 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:47.051 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:47.049+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:47.070 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:47.070 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:47.070 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:47.070 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:47.071 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:47.071 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:47.079 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:47.087 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:47.087 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:47.105 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:47.105 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:47.124 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:47.124 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:47.129 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:47.127+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:47.143 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:47.143 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:47.160 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:47.160 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:47.177 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:47.177 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:47.186 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:47.186 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:47.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:47.240 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:47.239+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:47.302 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:47.302 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:47.310 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:47.317 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:47.318 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:21:47.329 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-05-08T03:21:46.866575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1140761502,"num_objects":1416,"num_object_clones":221,"num_object_copies":2832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1416,"num_whiteouts":719,"num_read":193039,"num_read_kb":1187412,"num_write":22655,"num_write_kb":750670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":612652156,"num_keys_recovered":712,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18685,"ondisk_log_size":18685,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":879772,"kb_used_data":630076,"kb_used_omap":6,"kb_used_meta":249593,"kb_avail":376607588,"statfs":{"total":386547056640,"available":385646170112,"internally_reserved":0,"allocated":645197824,"data_stored":936417552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6379,"internal_metadata":255584021},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-152001168,"num_objects":-188,"num_object_clones":-53,"num_object_copies":-376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-188,"num_whiteouts":-50,"num_read":1409,"num_read_kb":1203,"num_write":258,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018272"},"pg_stats":[{"pgid":"3.1f","version":"145'349","reported_seq":3224,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.867265+0000","last_change":"2024-05-08T03:21:45.867265+0000","last_active":"2024-05-08T03:21:45.867265+0000","last_peered":"2024-05-08T03:21:45.867265+0000","last_clean":"2024-05-08T03:21:45.867265+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:21:45.867265+0000","last_undegraded":"2024-05-08T03:21:45.867265+0000","last_fullsized":"2024-05-08T03:21:45.867265+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:02:44.117852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031978520000000002,"stat_sum":{"num_bytes":34679537,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":2898,"num_read_kb":31303,"num_write":380,"num_write_kb":17426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1e","version":"145'335","reported_seq":3507,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.865710+0000","last_change":"2024-05-08T03:21:45.865981+0000","last_active":"2024-05-08T03:21:45.865710+0000","last_peered":"2024-05-08T03:21:45.865710+0000","last_clean":"2024-05-08T03:21:45.865710+0000","last_became_active":"2024-05-08T03:20:50.783719+0000","last_became_peered":"2024-05-08T03:20:50.783719+0000","last_unstale":"2024-05-08T03:21:45.865710+0000","last_undegraded":"2024-05-08T03:21:45.865710+0000","last_fullsized":"2024-05-08T03:21:45.865710+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:41.893373+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0024272450000000002,"stat_sum":{"num_bytes":21452273,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":29,"num_read":3234,"num_read_kb":10708,"num_write":366,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1d","version":"145'491","reported_seq":6973,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.875545+0000","last_change":"2024-05-08T03:21:45.875545+0000","last_active":"2024-05-08T03:21:45.875545+0000","last_peered":"2024-05-08T03:21:45.875545+0000","last_clean":"2024-05-08T03:21:45.875545+0000","last_became_active":"2024-05-08T03:20:39.773251+0000","last_became_peered":"2024-05-08T03:20:39.773251+0000","last_unstale":"2024-05-08T03:21:45.875545+0000","last_undegraded":"2024-05-08T03:21:45.875545+0000","last_fullsized":"2024-05-08T03:21:45.875545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:23.735987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061441100000000004,"stat_sum":{"num_bytes":54555280,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":6961,"num_read_kb":45402,"num_write":556,"num_write_kb":14719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1c","version":"145'986","reported_seq":4216,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.912469+0000","last_change":"2024-05-08T03:21:45.874959+0000","last_active":"2024-05-08T03:21:45.912469+0000","last_peered":"2024-05-08T03:21:45.912469+0000","last_clean":"2024-05-08T03:21:45.912469+0000","last_became_active":"2024-05-08T03:20:40.779478+0000","last_became_peered":"2024-05-08T03:20:40.779478+0000","last_unstale":"2024-05-08T03:21:45.912469+0000","last_undegraded":"2024-05-08T03:21:45.912469+0000","last_fullsized":"2024-05-08T03:21:45.912469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:10.303991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057616250000000003,"stat_sum":{"num_bytes":40332787,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":5795,"num_read_kb":52307,"num_write":1385,"num_write_kb":52325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1b","version":"143'550","reported_seq":4840,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.867533+0000","last_change":"2024-05-08T03:21:45.867922+0000","last_active":"2024-05-08T03:21:45.867533+0000","last_peered":"2024-05-08T03:21:45.867533+0000","last_clean":"2024-05-08T03:21:45.867533+0000","last_became_active":"2024-05-08T03:21:20.924394+0000","last_became_peered":"2024-05-08T03:21:20.924394+0000","last_unstale":"2024-05-08T03:21:45.867533+0000","last_undegraded":"2024-05-08T03:21:45.867533+0000","last_fullsized":"2024-05-08T03:21:45.867533+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:59:40.041091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030905289999999998,"stat_sum":{"num_bytes":26883872,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":5868,"num_read_kb":26423,"num_write":649,"num_write_kb":15961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7457235,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1a","version":"145'573","reported_seq":5768,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.967391+0000","last_change":"2024-05-08T03:21:45.870564+0000","last_active":"2024-05-08T03:21:45.967391+0000","last_peered":"2024-05-08T03:21:45.967391+0000","last_clean":"2024-05-08T03:21:45.967391+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:21:45.967391+0000","last_undegraded":"2024-05-08T03:21:45.967391+0000","last_fullsized":"2024-05-08T03:21:45.967391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:30:35.573675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00185991,"stat_sum":{"num_bytes":40440960,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":22,"num_read":5826,"num_read_kb":44844,"num_write":763,"num_write_kb":52877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.19","version":"145'1139","reported_seq":4687,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.937650+0000","last_change":"2024-05-08T03:21:45.868396+0000","last_active":"2024-05-08T03:21:45.937650+0000","last_peered":"2024-05-08T03:21:45.937650+0000","last_clean":"2024-05-08T03:21:45.937650+0000","last_became_active":"2024-05-08T03:20:51.780774+0000","last_became_peered":"2024-05-08T03:20:51.780774+0000","last_unstale":"2024-05-08T03:21:45.937650+0000","last_undegraded":"2024-05-08T03:21:45.937650+0000","last_fullsized":"2024-05-08T03:21:45.937650+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.694479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033257909999999998,"stat_sum":{"num_bytes":35706831,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":7329,"num_read_kb":24640,"num_write":1791,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.18","version":"145'560","reported_seq":9525,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.868576+0000","last_change":"2024-05-08T03:21:45.868576+0000","last_active":"2024-05-08T03:21:45.868576+0000","last_peered":"2024-05-08T03:21:45.868576+0000","last_clean":"2024-05-08T03:21:45.868576+0000","last_became_active":"2024-05-08T03:20:50.777845+0000","last_became_peered":"2024-05-08T03:20:50.777845+0000","last_unstale":"2024-05-08T03:21:45.868576+0000","last_undegraded":"2024-05-08T03:21:45.868576+0000","last_fullsized":"2024-05-08T03:21:45.868576+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:16:10.182766+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0040675520000000003,"stat_sum":{"num_bytes":61875224,"num_objects":50,"num_object_clones":5,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":25,"num_read":10894,"num_read_kb":61041,"num_write":531,"num_write_kb":16272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":18070131,"num_keys_recovered":60,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.17","version":"142'405","reported_seq":5191,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.868955+0000","last_change":"2024-05-08T03:21:45.869397+0000","last_active":"2024-05-08T03:21:45.868955+0000","last_peered":"2024-05-08T03:21:45.868955+0000","last_clean":"2024-05-08T03:21:45.868955+0000","last_became_active":"2024-05-08T03:20:50.780815+0000","last_became_peered":"2024-05-08T03:20:50.780815+0000","last_unstale":"2024-05-08T03:21:45.868955+0000","last_undegraded":"2024-05-08T03:21:45.868955+0000","last_fullsized":"2024-05-08T03:21:45.868955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:51.914248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039890519999999999,"stat_sum":{"num_bytes":24815839,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":19,"num_read":5540,"num_read_kb":25701,"num_write":410,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.16","version":"145'261","reported_seq":3484,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.868896+0000","last_change":"2024-05-08T03:21:45.868896+0000","last_active":"2024-05-08T03:21:45.868896+0000","last_peered":"2024-05-08T03:21:45.868896+0000","last_clean":"2024-05-08T03:21:45.868896+0000","last_became_active":"2024-05-08T03:20:51.783155+0000","last_became_peered":"2024-05-08T03:20:51.783155+0000","last_unstale":"2024-05-08T03:21:45.868896+0000","last_undegraded":"2024-05-08T03:21:45.868896+0000","last_fullsized":"2024-05-08T03:21:45.868896+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:37.473050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036651330000000001,"stat_sum":{"num_bytes":74362580,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":24,"num_read":2599,"num_read_kb":61854,"num_write":240,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":54103866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.15","version":"144'384","reported_seq":7521,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.873857+0000","last_change":"2024-05-08T03:21:45.874370+0000","last_active":"2024-05-08T03:21:45.873857+0000","last_peered":"2024-05-08T03:21:45.873857+0000","last_clean":"2024-05-08T03:21:45.873857+0000","last_became_active":"2024-05-08T03:20:50.783826+0000","last_became_peered":"2024-05-08T03:20:50.783826+0000","last_unstale":"2024-05-08T03:21:45.873857+0000","last_undegraded":"2024-05-08T03:21:45.873857+0000","last_fullsized":"2024-05-08T03:21:45.873857+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:29.021702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052215889999999996,"stat_sum":{"num_bytes":41556664,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":7013,"num_read_kb":37603,"num_write":357,"num_write_kb":8167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6817760,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.14","version":"145'822","reported_seq":5241,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.873801+0000","last_change":"2024-05-08T03:21:45.873801+0000","last_active":"2024-05-08T03:21:45.873801+0000","last_peered":"2024-05-08T03:21:45.873801+0000","last_clean":"2024-05-08T03:21:45.873801+0000","last_became_active":"2024-05-08T03:20:37.788932+0000","last_became_peered":"2024-05-08T03:20:37.788932+0000","last_unstale":"2024-05-08T03:21:45.873801+0000","last_undegraded":"2024-05-08T03:21:45.873801+0000","last_fullsized":"2024-05-08T03:21:45.873801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:09:15.395806+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047611429999999998,"stat_sum":{"num_bytes":11720236,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":7250,"num_read_kb":22534,"num_write":1006,"num_write_kb":9490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":2507487,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.863481+0000","last_change":"2024-05-08T03:20:47.820453+0000","last_active":"2024-05-08T03:21:45.863481+0000","last_peered":"2024-05-08T03:21:45.863481+0000","last_clean":"2024-05-08T03:21:45.863481+0000","last_became_active":"2024-05-08T03:20:47.820180+0000","last_became_peered":"2024-05-08T03:20:47.820180+0000","last_unstale":"2024-05-08T03:21:45.863481+0000","last_undegraded":"2024-05-08T03:21:45.863481+0000","last_fullsized":"2024-05-08T03:21:45.863481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:33.427981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"145'431","reported_seq":4549,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.987165+0000","last_change":"2024-05-08T03:21:45.868138+0000","last_active":"2024-05-08T03:21:45.987165+0000","last_peered":"2024-05-08T03:21:45.987165+0000","last_clean":"2024-05-08T03:21:45.987165+0000","last_became_active":"2024-05-08T03:20:50.781137+0000","last_became_peered":"2024-05-08T03:20:50.781137+0000","last_unstale":"2024-05-08T03:21:45.987165+0000","last_undegraded":"2024-05-08T03:21:45.987165+0000","last_fullsized":"2024-05-08T03:21:45.987165+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:32.701634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038505000000000002,"stat_sum":{"num_bytes":6160692,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":4090,"num_read_kb":14197,"num_write":422,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.868979+0000","last_change":"2024-05-08T03:20:47.821931+0000","last_active":"2024-05-08T03:21:45.868979+0000","last_peered":"2024-05-08T03:21:45.868979+0000","last_clean":"2024-05-08T03:21:45.868979+0000","last_became_active":"2024-05-08T03:20:47.821642+0000","last_became_peered":"2024-05-08T03:20:47.821642+0000","last_unstale":"2024-05-08T03:21:45.868979+0000","last_undegraded":"2024-05-08T03:21:45.868979+0000","last_fullsized":"2024-05-08T03:21:45.868979+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:42.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"145'768","reported_seq":4922,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.869110+0000","last_change":"2024-05-08T03:21:45.869110+0000","last_active":"2024-05-08T03:21:45.869110+0000","last_peered":"2024-05-08T03:21:45.869110+0000","last_clean":"2024-05-08T03:21:45.869110+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:21:45.869110+0000","last_undegraded":"2024-05-08T03:21:45.869110+0000","last_fullsized":"2024-05-08T03:21:45.869110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:26:07.172814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038240290000000001,"stat_sum":{"num_bytes":22854920,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":6996,"num_read_kb":34020,"num_write":957,"num_write_kb":18512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.5","version":"0'0","reported_seq":136,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.869318+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:21:45.869318+0000","last_peered":"2024-05-08T03:21:45.869318+0000","last_clean":"2024-05-08T03:21:45.869318+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:21:45.869318+0000","last_undegraded":"2024-05-08T03:21:45.869318+0000","last_fullsized":"2024-05-08T03:21:45.869318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"145'302","reported_seq":5502,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.875110+0000","last_change":"2024-05-08T03:21:45.875110+0000","last_active":"2024-05-08T03:21:45.875110+0000","last_peered":"2024-05-08T03:21:45.875110+0000","last_clean":"2024-05-08T03:21:45.875110+0000","last_became_active":"2024-05-08T03:20:50.783297+0000","last_became_peered":"2024-05-08T03:20:50.783297+0000","last_unstale":"2024-05-08T03:21:45.875110+0000","last_undegraded":"2024-05-08T03:21:45.875110+0000","last_fullsized":"2024-05-08T03:21:45.875110+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:47.512928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005906031,"stat_sum":{"num_bytes":4002158,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":4508,"num_read_kb":31383,"num_write":320,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":13300321,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":138,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.866202+0000","last_change":"2024-05-08T03:21:20.923976+0000","last_active":"2024-05-08T03:21:45.866202+0000","last_peered":"2024-05-08T03:21:45.866202+0000","last_clean":"2024-05-08T03:21:45.866202+0000","last_became_active":"2024-05-08T03:21:20.923726+0000","last_became_peered":"2024-05-08T03:21:20.923726+0000","last_unstale":"2024-05-08T03:21:45.866202+0000","last_undegraded":"2024-05-08T03:21:45.866202+0000","last_fullsized":"2024-05-08T03:21:45.866202+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:52.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"145'969","reported_seq":4771,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.942413+0000","last_change":"2024-05-08T03:21:45.871188+0000","last_active":"2024-05-08T03:21:45.942413+0000","last_peered":"2024-05-08T03:21:45.942413+0000","last_clean":"2024-05-08T03:21:45.942413+0000","last_became_active":"2024-05-08T03:20:38.806015+0000","last_became_peered":"2024-05-08T03:20:38.806015+0000","last_unstale":"2024-05-08T03:21:45.942413+0000","last_undegraded":"2024-05-08T03:21:45.942413+0000","last_fullsized":"2024-05-08T03:21:45.942413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:27.826222+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074609200000000002,"stat_sum":{"num_bytes":25464273,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":6057,"num_read_kb":34873,"num_write":1400,"num_write_kb":41081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":44040868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1","version":"145'334","reported_seq":4076,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.869976+0000","last_change":"2024-05-08T03:21:45.869976+0000","last_active":"2024-05-08T03:21:45.869976+0000","last_peered":"2024-05-08T03:21:45.869976+0000","last_clean":"2024-05-08T03:21:45.869976+0000","last_became_active":"2024-05-08T03:20:50.783285+0000","last_became_peered":"2024-05-08T03:20:50.783285+0000","last_unstale":"2024-05-08T03:21:45.869976+0000","last_undegraded":"2024-05-08T03:21:45.869976+0000","last_fullsized":"2024-05-08T03:21:45.869976+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:36:31.529048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427413,"stat_sum":{"num_bytes":7304940,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":2974,"num_read_kb":19727,"num_write":354,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14979418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"2.0","version":"18'117","reported_seq":414,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.864933+0000","last_change":"2024-05-08T03:20:50.784319+0000","last_active":"2024-05-08T03:21:45.864933+0000","last_peered":"2024-05-08T03:21:45.864933+0000","last_clean":"2024-05-08T03:21:45.864933+0000","last_became_active":"2024-05-08T03:20:50.783795+0000","last_became_peered":"2024-05-08T03:20:50.783795+0000","last_unstale":"2024-05-08T03:21:45.864933+0000","last_undegraded":"2024-05-08T03:21:45.864933+0000","last_fullsized":"2024-05-08T03:21:45.864933+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:15.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":269,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.865078+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:21:45.865078+0000","last_peered":"2024-05-08T03:21:45.865078+0000","last_clean":"2024-05-08T03:21:45.865078+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:21:45.865078+0000","last_undegraded":"2024-05-08T03:21:45.865078+0000","last_fullsized":"2024-05-08T03:21:45.865078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"144'468","reported_seq":7082,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.865050+0000","last_change":"2024-05-08T03:21:45.866129+0000","last_active":"2024-05-08T03:21:45.865050+0000","last_peered":"2024-05-08T03:21:45.865050+0000","last_clean":"2024-05-08T03:21:45.865050+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:21:45.865050+0000","last_undegraded":"2024-05-08T03:21:45.865050+0000","last_fullsized":"2024-05-08T03:21:45.865050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:36.323339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001391198,"stat_sum":{"num_bytes":15238619,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":7208,"num_read_kb":29295,"num_write":468,"num_write_kb":32552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.2","version":"0'0","reported_seq":136,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.868726+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:21:45.868726+0000","last_peered":"2024-05-08T03:21:45.868726+0000","last_clean":"2024-05-08T03:21:45.868726+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:21:45.868726+0000","last_undegraded":"2024-05-08T03:21:45.868726+0000","last_fullsized":"2024-05-08T03:21:45.868726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"145'878","reported_seq":8737,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.960616+0000","last_change":"2024-05-08T03:21:45.871115+0000","last_active":"2024-05-08T03:21:45.960616+0000","last_peered":"2024-05-08T03:21:45.960616+0000","last_clean":"2024-05-08T03:21:45.960616+0000","last_became_active":"2024-05-08T03:20:40.780249+0000","last_became_peered":"2024-05-08T03:20:40.780249+0000","last_unstale":"2024-05-08T03:21:45.960616+0000","last_undegraded":"2024-05-08T03:21:45.960616+0000","last_fullsized":"2024-05-08T03:21:45.960616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:19:35.795479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021431649999999998,"stat_sum":{"num_bytes":40655336,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":21,"num_read":11402,"num_read_kb":55636,"num_write":1086,"num_write_kb":28070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.1","version":"0'0","reported_seq":269,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.863235+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:21:45.863235+0000","last_peered":"2024-05-08T03:21:45.863235+0000","last_clean":"2024-05-08T03:21:45.863235+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:21:45.863235+0000","last_undegraded":"2024-05-08T03:21:45.863235+0000","last_fullsized":"2024-05-08T03:21:45.863235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"143'827","reported_seq":9734,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.868731+0000","last_change":"2024-05-08T03:21:45.869307+0000","last_active":"2024-05-08T03:21:45.868731+0000","last_peered":"2024-05-08T03:21:45.868731+0000","last_clean":"2024-05-08T03:21:45.868731+0000","last_became_active":"2024-05-08T03:20:52.799061+0000","last_became_peered":"2024-05-08T03:20:52.799061+0000","last_unstale":"2024-05-08T03:21:45.868731+0000","last_undegraded":"2024-05-08T03:21:45.868731+0000","last_fullsized":"2024-05-08T03:21:45.868731+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:21.352034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077534299999999995,"stat_sum":{"num_bytes":38060793,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":20,"num_read":10966,"num_read_kb":56008,"num_write":1099,"num_write_kb":16983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":191,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.864571+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:21:45.864571+0000","last_peered":"2024-05-08T03:21:45.864571+0000","last_clean":"2024-05-08T03:21:45.864571+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:21:45.864571+0000","last_undegraded":"2024-05-08T03:21:45.864571+0000","last_fullsized":"2024-05-08T03:21:45.864571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"145'509","reported_seq":4953,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.954550+0000","last_change":"2024-05-08T03:21:45.869868+0000","last_active":"2024-05-08T03:21:45.954550+0000","last_peered":"2024-05-08T03:21:45.954550+0000","last_clean":"2024-05-08T03:21:45.954550+0000","last_became_active":"2024-05-08T03:20:51.780777+0000","last_became_peered":"2024-05-08T03:20:51.780777+0000","last_unstale":"2024-05-08T03:21:45.954550+0000","last_undegraded":"2024-05-08T03:21:45.954550+0000","last_fullsized":"2024-05-08T03:21:45.954550+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:13.497672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042352559999999997,"stat_sum":{"num_bytes":6434848,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":6813,"num_read_kb":17649,"num_write":480,"num_write_kb":9915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5273218,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.9","version":"145'514","reported_seq":3816,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.873819+0000","last_change":"2024-05-08T03:21:45.873819+0000","last_active":"2024-05-08T03:21:45.873819+0000","last_peered":"2024-05-08T03:21:45.873819+0000","last_clean":"2024-05-08T03:21:45.873819+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:21:45.873819+0000","last_undegraded":"2024-05-08T03:21:45.873819+0000","last_fullsized":"2024-05-08T03:21:45.873819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:03:20.457711+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050801010000000001,"stat_sum":{"num_bytes":76521941,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":24,"num_read":4254,"num_read_kb":26398,"num_write":652,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.a","version":"145'569","reported_seq":2180,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.871681+0000","last_change":"2024-05-08T03:21:45.871681+0000","last_active":"2024-05-08T03:21:45.871681+0000","last_peered":"2024-05-08T03:21:45.871681+0000","last_clean":"2024-05-08T03:21:45.871681+0000","last_became_active":"2024-05-08T03:20:39.775306+0000","last_became_peered":"2024-05-08T03:20:39.775306+0000","last_unstale":"2024-05-08T03:21:45.871681+0000","last_undegraded":"2024-05-08T03:21:45.871681+0000","last_fullsized":"2024-05-08T03:21:45.871681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:14:11.776830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026638550000000001,"stat_sum":{"num_bytes":23773269,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":3186,"num_read_kb":10389,"num_write":616,"num_write_kb":5635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.b","version":"145'428","reported_seq":7490,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.943394+0000","last_change":"2024-05-08T03:21:45.866956+0000","last_active":"2024-05-08T03:21:45.943394+0000","last_peered":"2024-05-08T03:21:45.943394+0000","last_clean":"2024-05-08T03:21:45.943394+0000","last_became_active":"2024-05-08T03:21:20.923471+0000","last_became_peered":"2024-05-08T03:21:20.923471+0000","last_unstale":"2024-05-08T03:21:45.943394+0000","last_undegraded":"2024-05-08T03:21:45.943394+0000","last_fullsized":"2024-05-08T03:21:45.943394+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:57.383068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002176763,"stat_sum":{"num_bytes":39232002,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":6760,"num_read_kb":58888,"num_write":453,"num_write_kb":18516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":35463562,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.c","version":"145'772","reported_seq":5022,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.950300+0000","last_change":"2024-05-08T03:21:45.869514+0000","last_active":"2024-05-08T03:21:45.950300+0000","last_peered":"2024-05-08T03:21:45.950300+0000","last_clean":"2024-05-08T03:21:45.950300+0000","last_became_active":"2024-05-08T03:20:52.802662+0000","last_became_peered":"2024-05-08T03:20:52.802662+0000","last_unstale":"2024-05-08T03:21:45.950300+0000","last_undegraded":"2024-05-08T03:21:45.950300+0000","last_fullsized":"2024-05-08T03:21:45.950300+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:40.491706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041064580000000003,"stat_sum":{"num_bytes":78328304,"num_objects":58,"num_object_clones":11,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":6837,"num_read_kb":23476,"num_write":1035,"num_write_kb":23621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.d","version":"145'1014","reported_seq":5305,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.939594+0000","last_change":"2024-05-08T03:21:45.874437+0000","last_active":"2024-05-08T03:21:45.939594+0000","last_peered":"2024-05-08T03:21:45.939594+0000","last_clean":"2024-05-08T03:21:45.939594+0000","last_became_active":"2024-05-08T03:20:51.783022+0000","last_became_peered":"2024-05-08T03:20:51.783022+0000","last_unstale":"2024-05-08T03:21:45.939594+0000","last_undegraded":"2024-05-08T03:21:45.939594+0000","last_fullsized":"2024-05-08T03:21:45.939594+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1014,"log_dups_size":0,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:40.069120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052744569999999998,"stat_sum":{"num_bytes":56044030,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":23,"num_read":8992,"num_read_kb":74869,"num_write":1315,"num_write_kb":49359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":48212116,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.e","version":"145'611","reported_seq":4545,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.878775+0000","last_change":"2024-05-08T03:21:45.878956+0000","last_active":"2024-05-08T03:21:45.878775+0000","last_peered":"2024-05-08T03:21:45.878775+0000","last_clean":"2024-05-08T03:21:45.878775+0000","last_became_active":"2024-05-08T03:20:40.779118+0000","last_became_peered":"2024-05-08T03:20:40.779118+0000","last_unstale":"2024-05-08T03:21:45.878775+0000","last_undegraded":"2024-05-08T03:21:45.878775+0000","last_fullsized":"2024-05-08T03:21:45.878775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:12:45.144251+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0094149660000000003,"stat_sum":{"num_bytes":53510216,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":23,"num_read":4631,"num_read_kb":56936,"num_write":802,"num_write_kb":30706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":51071767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.f","version":"145'569","reported_seq":3984,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.984402+0000","last_change":"2024-05-08T03:21:45.877312+0000","last_active":"2024-05-08T03:21:45.984402+0000","last_peered":"2024-05-08T03:21:45.984402+0000","last_clean":"2024-05-08T03:21:45.984402+0000","last_became_active":"2024-05-08T03:20:38.805719+0000","last_became_peered":"2024-05-08T03:20:38.805719+0000","last_unstale":"2024-05-08T03:21:45.984402+0000","last_undegraded":"2024-05-08T03:21:45.984402+0000","last_fullsized":"2024-05-08T03:21:45.984402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:43:38.850331+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0078334300000000006,"stat_sum":{"num_bytes":12207535,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":3982,"num_read_kb":15803,"num_write":746,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":10326037,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.10","version":"145'494","reported_seq":4938,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.963483+0000","last_change":"2024-05-08T03:21:45.866814+0000","last_active":"2024-05-08T03:21:45.963483+0000","last_peered":"2024-05-08T03:21:45.963483+0000","last_clean":"2024-05-08T03:21:45.963483+0000","last_became_active":"2024-05-08T03:20:51.782569+0000","last_became_peered":"2024-05-08T03:20:51.782569+0000","last_unstale":"2024-05-08T03:21:45.963483+0000","last_undegraded":"2024-05-08T03:21:45.963483+0000","last_fullsized":"2024-05-08T03:21:45.963483+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:15:22.761444+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003381172,"stat_sum":{"num_bytes":56432384,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":5994,"num_read_kb":38231,"num_write":467,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.11","version":"145'484","reported_seq":5951,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.986576+0000","last_change":"2024-05-08T03:21:45.869966+0000","last_active":"2024-05-08T03:21:45.986576+0000","last_peered":"2024-05-08T03:21:45.986576+0000","last_clean":"2024-05-08T03:21:45.986576+0000","last_became_active":"2024-05-08T03:20:38.805730+0000","last_became_peered":"2024-05-08T03:20:38.805730+0000","last_unstale":"2024-05-08T03:21:45.986576+0000","last_undegraded":"2024-05-08T03:21:45.986576+0000","last_fullsized":"2024-05-08T03:21:45.986576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:54.367854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014643600000000001,"stat_sum":{"num_bytes":52650958,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":21,"num_read":6970,"num_read_kb":65960,"num_write":468,"num_write_kb":56640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":60512583,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.12","version":"145'377","reported_seq":2885,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.867463+0000","last_change":"2024-05-08T03:21:45.867463+0000","last_active":"2024-05-08T03:21:45.867463+0000","last_peered":"2024-05-08T03:21:45.867463+0000","last_clean":"2024-05-08T03:21:45.867463+0000","last_became_active":"2024-05-08T03:20:47.824316+0000","last_became_peered":"2024-05-08T03:20:47.824316+0000","last_unstale":"2024-05-08T03:21:45.867463+0000","last_undegraded":"2024-05-08T03:21:45.867463+0000","last_fullsized":"2024-05-08T03:21:45.867463+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:05.721398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026686349999999999,"stat_sum":{"num_bytes":35426366,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":4162,"num_read_kb":29959,"num_write":400,"num_write_kb":31580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.13","version":"143'395","reported_seq":5364,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-05-08T03:21:45.866033+0000","last_change":"2024-05-08T03:21:45.866534+0000","last_active":"2024-05-08T03:21:45.866033+0000","last_peered":"2024-05-08T03:21:45.866033+0000","last_clean":"2024-05-08T03:21:45.866033+0000","last_became_active":"2024-05-08T03:21:20.922351+0000","last_became_peered":"2024-05-08T03:21:20.922351+0000","last_unstale":"2024-05-08T03:21:45.866033+0000","last_undegraded":"2024-05-08T03:21:45.866033+0000","last_fullsized":"2024-05-08T03:21:45.866033+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:33.762917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028123929999999998,"stat_sum":{"num_bytes":21485467,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":29,"num_read":4934,"num_read_kb":49255,"num_write":522,"num_write_kb":27657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":40436240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1140171134,"num_objects":1414,"num_object_clones":221,"num_object_copies":2828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1414,"num_whiteouts":719,"num_read":192923,"num_read_kb":1187312,"num_write":22496,"num_write_kb":748254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4136,"num_bytes_recovered":610881052,"num_keys_recovered":712,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":853934080,"data_stored":1259198134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18568,"ondisk_log_size":18568,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542178,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":190660,"kb_used_data":123376,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94181180,"statfs":{"total":96636764160,"available":96441528320,"internally_reserved":0,"allocated":126337024,"data_stored":184971827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":0,"seq":455266533386,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":277832,"kb_used_data":220264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94094008,"statfs":{"total":96636764160,"available":96352264192,"internally_reserved":0,"allocated":225550336,"data_stored":323779094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542180,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167536,"kb_used_data":114144,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94204304,"statfs":{"total":96636764160,"available":96465207296,"internally_reserved":0,"allocated":116883456,"data_stored":174840440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243744,"kb_used_data":172292,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94128096,"statfs":{"total":96636764160,"available":96387170304,"internally_reserved":0,"allocated":176427008,"data_stored":252826191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162537472,"data_stored":239841931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108863488,"data_stored":167713984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":135417856,"data_stored":199525057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":223813632,"data_stored":322937845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":117698560,"data_stored":177240922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:21:47.351 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:47.350+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:47.433 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:47.431+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:47.455 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:47.455 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:47.465 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:47.534 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:47.533+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:47.569 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:47.569 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:47.578 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:47.662 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:47.661+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:47.724 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:47.724 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:47.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:47.773 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:47.772+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:47.837 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:47.836+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:47.885 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:47.885 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:47.893 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:47.976 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:47.975+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:48.038 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:48.037+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:48.040 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.040 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.049 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:48.138 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:48.137+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:48.194 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.194 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:48.247 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:48.246+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:48.308 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.308 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:48.358 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:48.357+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:48.442 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:48.441+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:48.463 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.463 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.472 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:48.543 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:48.541+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:48.579 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.579 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:48.643 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:48.642+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:48.734 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.734 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.743 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:48.744 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:48.743+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:48.846 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:48.845+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:48.847 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.847 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.856 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:48.885 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:21:48.946 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:48.945+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:48.971 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:48.971 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:48.981 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:49.047 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:49.046+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:49.129 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:49.129 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:49.137 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:49.182 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:49.180+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:49.189 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:49.245 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:49.245 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:49.249 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:49.247+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:49.255 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:49.312 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:49.313 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:49.331 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:49.331 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:49.347 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:49.348 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:49.350 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:49.348+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:49.364 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:49.365 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:49.382 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:49.382 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:49.399 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:49.399 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:49.402 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:49.402 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:49.402 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:49.402 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:49.411 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:49.417 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:49.417 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:49.434 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:49.435 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:49.451 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:49.450+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:49.526 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:49.526 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:49.526 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:49.526 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:49.527 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:49.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:49.551 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:49.551+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:49.639 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.537038001, 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.640 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:49.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.643 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:21:49.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.535317531, 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.645 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:21:49.646 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:21:49.647 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:21:49.648 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53528409, 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.650 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.651 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:21:49.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53385835100001, 2024-05-08T03:21:49.654 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.658 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.533765737, 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.659 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.533255063, 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.663 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:21:49.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:21:49.666 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.533227555, 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:21:49.670 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.671 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53282112399999, 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.673 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.674 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.676 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.532365746, 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.677 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:49.678 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:21:49.679 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:21:49.680 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53183749900001, 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.682 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.683 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.684 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.685 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.531365001, 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.690 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.531286869, 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.691 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.692 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.693 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.694 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.531211447, 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.695 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:21:49.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:21:49.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.699 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.531134284, 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:49.700 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:21:49.701 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:21:49.702 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:21:49.703 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.531056761, 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.704 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.705 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.706 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.707 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.708 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53102984900001, 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.709 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.710 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.711 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.712 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53100863900001, 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.713 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.715 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.716 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.717 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.530988859, 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:49.718 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.719 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.720 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.530434333, 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.722 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.723 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.724 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.725 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:49.726 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 101.53041532100001, 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:49.727 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.728 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.729 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:49.730 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:49.731 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:49.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:49.733 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:49.732+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:49.780 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:21:49.834 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:49.833+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:49.852 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:49.852 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:49.852 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:49.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:49.934 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:49.933+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:49.979 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:49.979 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:49.979 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:49.979 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:49.979 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:49.988 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:50.036 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:50.035+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:50.095 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:50.095 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:50.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:50.095 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:50.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.095 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 111.096319416, 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 111.096251952, 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.101 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:21:50.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:21:50.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:21:50.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 111.03641761999999, 2024-05-08T03:21:50.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.96990960700001, 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.114 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:21:50.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:21:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.119 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.943371961, 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 109.915460193, 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.125 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:50.128 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.836879999999994, 2024-05-08T03:21:50.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.132 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832252624000006, 2024-05-08T03:21:50.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:50.137 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832231988999993, 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:21:50.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832209738000003, 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.144 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.145 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.146 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.147 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832189306999993, 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.148 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.149 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.150 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:21:50.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832169081000004, 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.152 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:21:50.153 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:21:50.154 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:21:50.155 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832148286999995, 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.156 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.157 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.158 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.159 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.832122272999996, 2024-05-08T03:21:50.160 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.163 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:50.164 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.652165353000001, 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:21:50.166 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:21:50.167 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.168 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.652146248999998, 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:50.169 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.170 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:21:50.171 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.172 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.651584674999995, 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.173 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.174 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.175 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.176 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.646370813999994, 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.177 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:21:50.178 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:21:50.179 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:21:50.180 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:21:50.182 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 81.646203473, 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:50.183 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:21:50.184 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:21:50.185 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:50.186 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.187 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 73.283449876000006, 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:21:50.188 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:50.189 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:21:50.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.191 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:50.192 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:50.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:50.194 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:50.193+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:50.295 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:50.293+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:50.300 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:50.300 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:50.309 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:50.396 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:50.394+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:50.414 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:50.414 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:50.414 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:50.414 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:50.414 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:50.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:50.497 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:50.495+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:50.531 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.755696465, 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:50.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:50.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:50.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:50.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.74107326, 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:50.543 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.740263980999998, 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.544 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:50.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.739324703999998, 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.737691634000001, 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:50.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.736253138000002, 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:50.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:50.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:50.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.736233249999998, 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:50.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:50.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.736209330999998, 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:50.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:50.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:50.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.577 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.735238150999997, 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:50.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.735226541999999, 2024-05-08T03:21:50.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:50.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.587 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.735216823999998, 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:50.592 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.735206362, 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:50.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.732629416000002, 2024-05-08T03:21:50.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:50.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:50.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.732599168999997, 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:50.602 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.732572152000003, 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.608 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:50.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:50.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:50.613 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.732553158999998, 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:50.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.732523100000002, 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.623 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.730738635999998, 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:50.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:50.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.730726812999997, 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.629 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:50.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:50.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:50.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 59.730714956, 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:50.634 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:50.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:50.639 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:50.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:50.641 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:50.639+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:50.742 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:50.741+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:50.747 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:50.747 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:50.747 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:50.747 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:50.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:50.843 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:50.841+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:50.906 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:50.906 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:50.906 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:50.907 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:50.907 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:50.916 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:50.943 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:50.942+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:51.039 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 134.82844413999999, 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:51.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:51.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.90871382899999, 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.049 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.90868211200001, 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:51.053 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.90865700800001, 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.908632203, 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.058 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.90860365899999, 2024-05-08T03:21:51.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.066 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.908581137, 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:51.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.070 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.908550797, 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:51.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 133.90852961600001, 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.076 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:51.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 120.09588768, 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.080 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.90721295900001, 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:51.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:51.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.885009616, 2024-05-08T03:21:51.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:51.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.88331349800001, 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.86360870999999, 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.101 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:51.106 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.86358475, 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:51.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:51.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:51.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:51.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.822993205, 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 110.810219308, 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.116 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:51.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:51.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:51.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 91.549480373999998, 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:51.120 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:21:51.124 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.191175975, 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:51.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:21:51.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:21:51.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 36.189557237999999, 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.132 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.133 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.133 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.133 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.133 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:51.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:51.135 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:51.133+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:51.235 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:51.234+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:51.282 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:51.282 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:51.282 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:51.282 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:51.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:51.336 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:51.335+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:51.395 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:51.395 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:51.395 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:51.403 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:51.438 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:51.436+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:51.446 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 139.633780422, 2024-05-08T03:21:51.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:51.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:51.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:51.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.516 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29183922999999, 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:51.520 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29182383099999, 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.521 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:51.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29179218900001, 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.525 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.529 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.291778857, 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:51.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.291769063, 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.534 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:51.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:51.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29175964999999, 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29175012900001, 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.542 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:51.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:51.546 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.291740788, 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.550 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29173157299999, 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29172221100001, 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:51.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29171277899999, 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:51.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29170336300001, 2024-05-08T03:21:51.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.291694345, 2024-05-08T03:21:51.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:51.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:51.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29168530199999, 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:51.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:51.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29167300899999, 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.584 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.291663742, 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.585 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.29164988400001, 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.589 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:51.593 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.291640591, 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 135.298626458, 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:51.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:51.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:51.602 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:51.603 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:51.604 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:51.604 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:51.605 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:51.605 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:51.605 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:51.603+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:51.606 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:51.606 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:51.622 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:51.622 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:51.639 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:51.639 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:51.656 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:51.656 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:51.673 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:51.674 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:51.691 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:51.692 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:51.704 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:51.704+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:51.753 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:51.753 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:51.762 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:51.805 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:21:51.806 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:51.804+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:51.907 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:51.905+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:51.914 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:51.914 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:51.924 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:52.008 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:52.006+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:52.038 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.038 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.047 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:52.109 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:52.108+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:52.155 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.155 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.164 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:52.210 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:52.209+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:52.268 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.268 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.277 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:52.311 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:52.310+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:52.332 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:52.332 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:52.332 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-08T03:21:52.333 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T03:21:52.384 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.384 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.395 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:52.438 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:52.437+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:52.504 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.504 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.512 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:52.511+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:52.513 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:52.614 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:52.613+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:52.632 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:52.632 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "epoch": 151, 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "fsid": "a7e973ac-16c0-41c5-817e-e3e8f10aad7b", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "created": "2024-05-08T03:18:59.968487+0000", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "modified": "2024-05-08T03:21:51.881849+0000", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "last_up_change": "2024-05-08T03:21:03.827528+0000", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "last_in_change": "2024-05-08T03:20:46.806790+0000", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_num": 5804032, 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_set": [ 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "nodeep-scrub", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "pglog_hardlimit", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "purged_snapdirs", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_deletes", 2024-05-08T03:21:52.633 INFO:teuthology.orchestra.run.smithi177.stdout: "sortbitwise" 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_version": 6, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_snapshot": "", 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_max": 3, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "max_osd": 8, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "require_min_compat_client": "luminous", 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "min_compat_client": "luminous", 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "require_osd_release": "squid", 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "allow_crimson": false, 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "pools": [ 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.634 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 1, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "rbd", 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:11.648360+0000", 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:21:52.635 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 8, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 8, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 8, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 8, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 8, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "17", 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:21:52.636 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:21:52.637 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:21:52.638 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 3, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 3, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.5, 2024-05-08T03:21:52.639 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.5, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.5, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.83005142211914062, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 0.82999998331069946, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 2, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": ".mgr", 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:12.206654+0000", 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 1, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool", 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:21:52.640 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 1, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 1, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 1, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 1, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 1, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:21:52.641 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "18", 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 0, 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 0, 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:21:52.642 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:21:52.643 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "options": { 2024-05-08T03:21:52.644 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_max": 32, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_min": 1 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": {} 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 8, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 8, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.25, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 2, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 2, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.83005142211914062, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 0.82999998331069946, 2024-05-08T03:21:52.645 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_name": "pool_client.0", 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "create_time": "2024-05-08T03:19:19.077332+0000", 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "flags": 8193, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "type": 1, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 2, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "min_size": 1, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_rule": 0, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_count": 0, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_target": 0, 2024-05-08T03:21:52.646 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "object_hash": 2, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_autoscale_mode": "off", 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num": 32, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num": 32, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_placement_num_target": 32, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_target": 32, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_num_pending": 32, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "last_pg_merge_meta": { 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "source_pgid": "0.0", 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "ready_epoch": 0, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_started": 0, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "last_epoch_clean": 0, 2024-05-08T03:21:52.647 INFO:teuthology.orchestra.run.smithi177.stdout: "source_version": "0'0", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "target_version": "0'0" 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "last_change": "151", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend": "0", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "auid": 0, 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_mode": "selfmanaged", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_seq": 49, 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "snap_epoch": 151, 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "pool_snaps": [], 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps": "[]", 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_bytes": 0, 2024-05-08T03:21:52.648 INFO:teuthology.orchestra.run.smithi177.stdout: "quota_max_objects": 0, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "tiers": [], 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "tier_of": -1, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "read_tier": -1, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "write_tier": -1, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_mode": "none", 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_bytes": 0, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "target_max_objects": 0, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_flush_age": 0, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_min_evict_age": 0, 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profile": "", 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_params": { 2024-05-08T03:21:52.649 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "none" 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_period": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_count": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "use_gmt_hitset": true, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "min_read_recency_for_promote": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "min_write_recency_for_promote": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "hit_set_search_last_n": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "grade_table": [], 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "stripe_width": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "expected_num_objects": 0, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "fast_read": false, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "options": {}, 2024-05-08T03:21:52.650 INFO:teuthology.orchestra.run.smithi177.stdout: "application_metadata": { 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "rbd": {} 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "read_balance": { 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "score_type": "Fair distribution", 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "score_acting": 3.7599999904632568, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "score_stable": 3.7599999904632568, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "optimal_score": 0.5, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_acting": 1.8799999952316284, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "raw_score_stable": 1.8799999952316284, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity_weighted": 0.83005142211914062, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity": 0.82999998331069946, 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.651 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "5bdb4f77-60d7-46da-a930-5450259e00b4", 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 40, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.652 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6808", 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6809", 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.653 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6810", 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6811", 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.654 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6814", 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6815", 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.655 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6812", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6813", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2599410006 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6809/2599410006", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6811/2599410006", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6815/2599410006", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6813/2599410006", 2024-05-08T03:21:52.656 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9", 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.657 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 120, 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6821", 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6823", 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.658 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6825", 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6827", 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.659 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6830", 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6831", 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.660 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6828", 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6829", 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3471641477 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6823/3471641477", 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6827/3471641477", 2024-05-08T03:21:52.661 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6831/3471641477", 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6829/3471641477", 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "c712ddbe-4915-4670-a683-4acdd11404e3", 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.662 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 120, 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6800", 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6801", 2024-05-08T03:21:52.663 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6802", 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6803", 2024-05-08T03:21:52.664 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6806", 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6807", 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.665 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6804", 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6805", 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2694847262 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.666 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6801/2694847262", 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6803/2694847262", 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6807/2694847262", 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6805/2694847262", 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "3e5c0168-06af-4e9a-8dd1-eb25895cb480", 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.667 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 98, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6816", 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6817", 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.670 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.670 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6818", 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6819", 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.671 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6824", 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6826", 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.672 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6820", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.177:6822", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 4093771454 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.177:6817/4093771454", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.177:6819/4093771454", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.177:6826/4093771454", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.177:6822/4093771454", 2024-05-08T03:21:52.673 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "6668ff75-4ff3-47a0-ba04-95ae6e233fe0", 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0.5313873291015625, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.674 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 99, 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6800", 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6801", 2024-05-08T03:21:52.675 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6802", 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6803", 2024-05-08T03:21:52.676 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6806", 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6807", 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.677 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6804", 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6805", 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1585905529 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.678 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6801/1585905529", 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6803/1585905529", 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6807/1585905529", 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6805/1585905529", 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "9e02c1b8-134f-43bf-917c-966ba4eac713", 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.679 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 14, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 58, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 106, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 45, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 61, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6816", 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6817", 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6818", 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6819", 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6822", 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6823", 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6820", 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.683 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6821", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 1469026711 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6817/1469026711", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6819/1469026711", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6823/1469026711", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6821/1469026711", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.684 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "38f1ad8a-9e66-4e02-bf65-0d93e046ad5b", 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 1, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 1, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 0.1090240478515625, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 120, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6808", 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6809", 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.686 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6810", 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6811", 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.687 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6814", 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6815", 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6812", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6813", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3787670496 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6809/3787670496", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6811/3787670496", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6815/3787670496", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6813/3787670496", 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.689 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "uuid": "bd284e7d-47ec-489a-bd56-d5710e1fad27", 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "up": 1, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "in": 0, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "weight": 0, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_affinity": 1, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_begin": 0, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "last_clean_end": 0, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "up_from": 14, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "up_thru": 31, 2024-05-08T03:21:52.690 INFO:teuthology.orchestra.run.smithi177.stdout: "down_at": 0, 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "lost_at": 0, 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addrs": { 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6824", 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6825", 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.691 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addrs": { 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6826", 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6827", 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addrs": { 2024-05-08T03:21:52.692 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6830", 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6831", 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addrs": { 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: "addrvec": [ 2024-05-08T03:21:52.693 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v2", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6828", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "172.21.15.188:6829", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 405203122 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "public_addr": "172.21.15.188:6825/405203122", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "cluster_addr": "172.21.15.188:6827/405203122", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_back_addr": "172.21.15.188:6831/405203122", 2024-05-08T03:21:52.694 INFO:teuthology.orchestra.run.smithi177.stdout: "heartbeat_front_addr": "172.21.15.188:6829/405203122", 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "state": [ 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "exists", 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "up" 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_xinfo": [ 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 0, 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:52.695 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.148758+0000", 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 1, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.170231+0000", 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 2, 2024-05-08T03:21:52.696 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.129002+0000", 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 3, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.697 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.179213+0000", 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 4, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.075940+0000", 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.698 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 5, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122695+0000", 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 61 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 6, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.699 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 0, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:10.421746+0000", 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "osd": 7, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "down_stamp": "0.000000", 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_probability": 0, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "laggy_interval": 0, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "features": 4540701547738038271, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "old_weight": 65536, 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "last_purged_snaps_scrub": "2024-05-08T03:19:08.122924+0000", 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "dead_epoch": 0 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.700 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap": [ 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.19", 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "osds": [ 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: 3, 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: 4 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_items": [ 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pgid": "3.6", 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "mappings": [ 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "from": 1, 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: "to": 2 2024-05-08T03:21:52.701 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_upmap_primaries": [], 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "pg_temp": [], 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "primary_temp": [], 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklist": {}, 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "range_blocklist": {}, 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "erasure_code_profiles": { 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "default": { 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "crush-failure-domain": "osd", 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "k": "2", 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "m": "1", 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "plugin": "jerasure", 2024-05-08T03:21:52.702 INFO:teuthology.orchestra.run.smithi177.stdout: "technique": "reed_sol_van" 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: "removed_snaps_queue": [ 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: "snaps": [ 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: "begin": 20, 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: "length": 6 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: "new_removed_snaps": [ 2024-05-08T03:21:52.703 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "pool": 3, 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "snaps": [ 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "begin": 20, 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "length": 1 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: ], 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "new_purged_snaps": [], 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "crush_node_flags": {}, 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "device_class_flags": {}, 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode": { 2024-05-08T03:21:52.704 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_enabled": false, 2024-05-08T03:21:52.705 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_bucket_count": 0, 2024-05-08T03:21:52.705 INFO:teuthology.orchestra.run.smithi177.stdout: "degraded_stretch_mode": 0, 2024-05-08T03:21:52.705 INFO:teuthology.orchestra.run.smithi177.stdout: "recovering_stretch_mode": 0, 2024-05-08T03:21:52.705 INFO:teuthology.orchestra.run.smithi177.stdout: "stretch_mode_bucket": 0 2024-05-08T03:21:52.705 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:52.705 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:52.707 INFO:tasks.thrashosds.thrasher:j is {'epoch': 151, 'fsid': 'a7e973ac-16c0-41c5-817e-e3e8f10aad7b', 'created': '2024-05-08T03:18:59.968487+0000', 'modified': '2024-05-08T03:21:51.881849+0000', 'last_up_change': '2024-05-08T03:21:03.827528+0000', 'last_in_change': '2024-05-08T03:20:46.806790+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T03:19:11.648360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8300514221191406, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T03:19:12.206654+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300514221191406, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T03:19:19.077332+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 49, 'snap_epoch': 151, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.759999990463257, 'score_stable': 3.759999990463257, 'optimal_score': 0.5, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.8300514221191406, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '5bdb4f77-60d7-46da-a930-5450259e00b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6808', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6809', 'nonce': 2599410006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6810', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6811', 'nonce': 2599410006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6814', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6815', 'nonce': 2599410006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6812', 'nonce': 2599410006}, {'type': 'v1', 'addr': '172.21.15.177:6813', 'nonce': 2599410006}]}, 'public_addr': '172.21.15.177:6809/2599410006', 'cluster_addr': '172.21.15.177:6811/2599410006', 'heartbeat_back_addr': '172.21.15.177:6815/2599410006', 'heartbeat_front_addr': '172.21.15.177:6813/2599410006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6821', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6823', 'nonce': 3471641477}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6825', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6827', 'nonce': 3471641477}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6830', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6831', 'nonce': 3471641477}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6828', 'nonce': 3471641477}, {'type': 'v1', 'addr': '172.21.15.177:6829', 'nonce': 3471641477}]}, 'public_addr': '172.21.15.177:6823/3471641477', 'cluster_addr': '172.21.15.177:6827/3471641477', 'heartbeat_back_addr': '172.21.15.177:6831/3471641477', 'heartbeat_front_addr': '172.21.15.177:6829/3471641477', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c712ddbe-4915-4670-a683-4acdd11404e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6800', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6801', 'nonce': 2694847262}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6802', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6803', 'nonce': 2694847262}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6806', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6807', 'nonce': 2694847262}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6804', 'nonce': 2694847262}, {'type': 'v1', 'addr': '172.21.15.177:6805', 'nonce': 2694847262}]}, 'public_addr': '172.21.15.177:6801/2694847262', 'cluster_addr': '172.21.15.177:6803/2694847262', 'heartbeat_back_addr': '172.21.15.177:6807/2694847262', 'heartbeat_front_addr': '172.21.15.177:6805/2694847262', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3e5c0168-06af-4e9a-8dd1-eb25895cb480', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6816', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6817', 'nonce': 4093771454}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6818', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6819', 'nonce': 4093771454}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6824', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6826', 'nonce': 4093771454}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6820', 'nonce': 4093771454}, {'type': 'v1', 'addr': '172.21.15.177:6822', 'nonce': 4093771454}]}, 'public_addr': '172.21.15.177:6817/4093771454', 'cluster_addr': '172.21.15.177:6819/4093771454', 'heartbeat_back_addr': '172.21.15.177:6826/4093771454', 'heartbeat_front_addr': '172.21.15.177:6822/4093771454', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6668ff75-4ff3-47a0-ba04-95ae6e233fe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5313873291015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6800', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6801', 'nonce': 1585905529}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6802', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6803', 'nonce': 1585905529}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6806', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6807', 'nonce': 1585905529}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6804', 'nonce': 1585905529}, {'type': 'v1', 'addr': '172.21.15.188:6805', 'nonce': 1585905529}]}, 'public_addr': '172.21.15.188:6801/1585905529', 'cluster_addr': '172.21.15.188:6803/1585905529', 'heartbeat_back_addr': '172.21.15.188:6807/1585905529', 'heartbeat_front_addr': '172.21.15.188:6805/1585905529', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9e02c1b8-134f-43bf-917c-966ba4eac713', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 58, 'up_from': 106, 'up_thru': 45, 'down_at': 61, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6816', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6817', 'nonce': 1469026711}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6818', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6819', 'nonce': 1469026711}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6822', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6823', 'nonce': 1469026711}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6820', 'nonce': 1469026711}, {'type': 'v1', 'addr': '172.21.15.188:6821', 'nonce': 1469026711}]}, 'public_addr': '172.21.15.188:6817/1469026711', 'cluster_addr': '172.21.15.188:6819/1469026711', 'heartbeat_back_addr': '172.21.15.188:6823/1469026711', 'heartbeat_front_addr': '172.21.15.188:6821/1469026711', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '38f1ad8a-9e66-4e02-bf65-0d93e046ad5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1090240478515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6808', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6809', 'nonce': 3787670496}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6810', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6811', 'nonce': 3787670496}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6814', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6815', 'nonce': 3787670496}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6812', 'nonce': 3787670496}, {'type': 'v1', 'addr': '172.21.15.188:6813', 'nonce': 3787670496}]}, 'public_addr': '172.21.15.188:6809/3787670496', 'cluster_addr': '172.21.15.188:6811/3787670496', 'heartbeat_back_addr': '172.21.15.188:6815/3787670496', 'heartbeat_front_addr': '172.21.15.188:6813/3787670496', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd284e7d-47ec-489a-bd56-d5710e1fad27', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6824', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6825', 'nonce': 405203122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6826', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6827', 'nonce': 405203122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6830', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6831', 'nonce': 405203122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6828', 'nonce': 405203122}, {'type': 'v1', 'addr': '172.21.15.188:6829', 'nonce': 405203122}]}, 'public_addr': '172.21.15.188:6825/405203122', 'cluster_addr': '172.21.15.188:6827/405203122', 'heartbeat_back_addr': '172.21.15.188:6831/405203122', 'heartbeat_front_addr': '172.21.15.188:6829/405203122', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.148758+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.170231+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.129002+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.179213+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.075940+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122695+0000', 'dead_epoch': 61}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T03:19:10.421746+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-08T03:19:08.122924+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.19', 'osds': [3, 4]}], 'pg_upmap_items': [{'pgid': '3.6', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 6}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T03:21:52.707 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:21:52.708 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.708 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.708 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:52.714 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:52.713+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:52.815 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:52.813+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:52.858 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:52.858 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:52.869 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:52.897 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:21:52.919 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:52.917+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:53.017 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:53.016+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:53.023 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.023 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.033 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:53.086 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:21:53.086 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:21:53.097 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-05-08T03:21:52.901768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":997100504,"num_objects":1108,"num_object_clones":148,"num_object_copies":2216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1108,"num_whiteouts":706,"num_read":194339,"num_read_kb":1188524,"num_write":23047,"num_write_kb":750733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":612652156,"num_keys_recovered":712,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19244,"ondisk_log_size":19244,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":830104,"kb_used_data":575144,"kb_used_omap":6,"kb_used_meta":253753,"kb_avail":376657256,"statfs":{"total":386547056640,"available":385697030144,"internally_reserved":0,"allocated":588947456,"data_stored":865541654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6379,"internal_metadata":259843861},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-143660998,"num_objects":-308,"num_object_clones":-73,"num_object_copies":-616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-308,"num_whiteouts":-13,"num_read":1300,"num_read_kb":1112,"num_write":392,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023833"},"pg_stats":[{"pgid":"3.1f","version":"150'362","reported_seq":3293,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.053243+0000","last_change":"2024-05-08T03:21:50.885175+0000","last_active":"2024-05-08T03:21:51.053243+0000","last_peered":"2024-05-08T03:21:51.053243+0000","last_clean":"2024-05-08T03:21:51.053243+0000","last_became_active":"2024-05-08T03:19:56.988701+0000","last_became_peered":"2024-05-08T03:19:56.988701+0000","last_unstale":"2024-05-08T03:21:51.053243+0000","last_undegraded":"2024-05-08T03:21:51.053243+0000","last_fullsized":"2024-05-08T03:21:51.053243+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:07:26.660078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011080930000000001,"stat_sum":{"num_bytes":33716977,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":2966,"num_read_kb":31360,"num_write":392,"num_write_kb":17431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.1e","version":"150'340","reported_seq":3548,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.976986+0000","last_change":"2024-05-08T03:21:50.885708+0000","last_active":"2024-05-08T03:21:50.976986+0000","last_peered":"2024-05-08T03:21:50.976986+0000","last_clean":"2024-05-08T03:21:50.976986+0000","last_became_active":"2024-05-08T03:20:50.783719+0000","last_became_peered":"2024-05-08T03:20:50.783719+0000","last_unstale":"2024-05-08T03:21:50.976986+0000","last_undegraded":"2024-05-08T03:21:50.976986+0000","last_fullsized":"2024-05-08T03:21:50.976986+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:01:08.833066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001505969,"stat_sum":{"num_bytes":21228086,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":3236,"num_read_kb":10709,"num_write":369,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.1d","version":"150'507","reported_seq":7025,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.969473+0000","last_change":"2024-05-08T03:21:50.893776+0000","last_active":"2024-05-08T03:21:50.969473+0000","last_peered":"2024-05-08T03:21:50.969473+0000","last_clean":"2024-05-08T03:21:50.969473+0000","last_became_active":"2024-05-08T03:20:39.773251+0000","last_became_peered":"2024-05-08T03:20:39.773251+0000","last_unstale":"2024-05-08T03:21:50.969473+0000","last_undegraded":"2024-05-08T03:21:50.969473+0000","last_fullsized":"2024-05-08T03:21:50.969473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:30.403249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001618527,"stat_sum":{"num_bytes":37171820,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":6967,"num_read_kb":45405,"num_write":564,"num_write_kb":14719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.1c","version":"150'1002","reported_seq":4289,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.028530+0000","last_change":"2024-05-08T03:21:50.892291+0000","last_active":"2024-05-08T03:21:51.028530+0000","last_peered":"2024-05-08T03:21:51.028530+0000","last_clean":"2024-05-08T03:21:51.028530+0000","last_became_active":"2024-05-08T03:20:40.779478+0000","last_became_peered":"2024-05-08T03:20:40.779478+0000","last_unstale":"2024-05-08T03:21:51.028530+0000","last_undegraded":"2024-05-08T03:21:51.028530+0000","last_fullsized":"2024-05-08T03:21:51.028530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:44.692322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065640299999999996,"stat_sum":{"num_bytes":38449506,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":18,"num_read":5828,"num_read_kb":52338,"num_write":1402,"num_write_kb":52325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.1b","version":"150'569","reported_seq":4899,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.977172+0000","last_change":"2024-05-08T03:21:50.888950+0000","last_active":"2024-05-08T03:21:50.977172+0000","last_peered":"2024-05-08T03:21:50.977172+0000","last_clean":"2024-05-08T03:21:50.977172+0000","last_became_active":"2024-05-08T03:21:20.924394+0000","last_became_peered":"2024-05-08T03:21:20.924394+0000","last_unstale":"2024-05-08T03:21:50.977172+0000","last_undegraded":"2024-05-08T03:21:50.977172+0000","last_fullsized":"2024-05-08T03:21:50.977172+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:08.746737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029104069999999998,"stat_sum":{"num_bytes":24895536,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":22,"num_read":5889,"num_read_kb":26438,"num_write":663,"num_write_kb":15961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7457235,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.1a","version":"150'575","reported_seq":5840,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.037432+0000","last_change":"2024-05-08T03:21:50.894295+0000","last_active":"2024-05-08T03:21:51.037432+0000","last_peered":"2024-05-08T03:21:51.037432+0000","last_clean":"2024-05-08T03:21:51.037432+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:21:51.037432+0000","last_undegraded":"2024-05-08T03:21:51.037432+0000","last_fullsized":"2024-05-08T03:21:51.037432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:35:27.069407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001808446,"stat_sum":{"num_bytes":40133760,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":5861,"num_read_kb":44879,"num_write":765,"num_write_kb":52877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.19","version":"150'1143","reported_seq":4728,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.962114+0000","last_change":"2024-05-08T03:21:50.886951+0000","last_active":"2024-05-08T03:21:50.962114+0000","last_peered":"2024-05-08T03:21:50.962114+0000","last_clean":"2024-05-08T03:21:50.962114+0000","last_became_active":"2024-05-08T03:20:51.780774+0000","last_became_peered":"2024-05-08T03:20:51.780774+0000","last_unstale":"2024-05-08T03:21:50.962114+0000","last_undegraded":"2024-05-08T03:21:50.962114+0000","last_fullsized":"2024-05-08T03:21:50.962114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1143,"log_dups_size":0,"ondisk_log_size":1143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:20.696848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017202840000000001,"stat_sum":{"num_bytes":35329983,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":26,"num_read":7332,"num_read_kb":24641,"num_write":1795,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.18","version":"150'570","reported_seq":9582,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.026144+0000","last_change":"2024-05-08T03:21:50.888706+0000","last_active":"2024-05-08T03:21:51.026144+0000","last_peered":"2024-05-08T03:21:51.026144+0000","last_clean":"2024-05-08T03:21:51.026144+0000","last_became_active":"2024-05-08T03:20:50.777845+0000","last_became_peered":"2024-05-08T03:20:50.777845+0000","last_unstale":"2024-05-08T03:21:51.026144+0000","last_undegraded":"2024-05-08T03:21:51.026144+0000","last_fullsized":"2024-05-08T03:21:51.026144+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:39.196218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027945869999999999,"stat_sum":{"num_bytes":44418131,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":10931,"num_read_kb":61071,"num_write":539,"num_write_kb":16272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":18070131,"num_keys_recovered":60,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.17","version":"150'414","reported_seq":5236,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.960511+0000","last_change":"2024-05-08T03:21:50.887417+0000","last_active":"2024-05-08T03:21:50.960511+0000","last_peered":"2024-05-08T03:21:50.960511+0000","last_clean":"2024-05-08T03:21:50.960511+0000","last_became_active":"2024-05-08T03:20:50.780815+0000","last_became_peered":"2024-05-08T03:20:50.780815+0000","last_unstale":"2024-05-08T03:21:50.960511+0000","last_undegraded":"2024-05-08T03:21:50.960511+0000","last_fullsized":"2024-05-08T03:21:50.960511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:55.708829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020528349999999998,"stat_sum":{"num_bytes":7300473,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":5542,"num_read_kb":25702,"num_write":417,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.16","version":"149'274","reported_seq":3530,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.888769+0000","last_change":"2024-05-08T03:21:50.889165+0000","last_active":"2024-05-08T03:21:50.888769+0000","last_peered":"2024-05-08T03:21:50.888769+0000","last_clean":"2024-05-08T03:21:50.888769+0000","last_became_active":"2024-05-08T03:20:51.783155+0000","last_became_peered":"2024-05-08T03:20:51.783155+0000","last_unstale":"2024-05-08T03:21:50.888769+0000","last_undegraded":"2024-05-08T03:21:50.888769+0000","last_fullsized":"2024-05-08T03:21:50.888769+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:01.201086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030724709999999998,"stat_sum":{"num_bytes":73252564,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":24,"num_read":2599,"num_read_kb":61854,"num_write":245,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":54103866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.15","version":"150'392","reported_seq":7564,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.982673+0000","last_change":"2024-05-08T03:21:50.894053+0000","last_active":"2024-05-08T03:21:50.982673+0000","last_peered":"2024-05-08T03:21:50.982673+0000","last_clean":"2024-05-08T03:21:50.982673+0000","last_became_active":"2024-05-08T03:20:50.783826+0000","last_became_peered":"2024-05-08T03:20:50.783826+0000","last_unstale":"2024-05-08T03:21:50.982673+0000","last_undegraded":"2024-05-08T03:21:50.982673+0000","last_fullsized":"2024-05-08T03:21:50.982673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:34.096256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178393,"stat_sum":{"num_bytes":24222392,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":29,"num_read":7013,"num_read_kb":37603,"num_write":365,"num_write_kb":8167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":6817760,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.14","version":"150'834","reported_seq":5287,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.979894+0000","last_change":"2024-05-08T03:21:50.893949+0000","last_active":"2024-05-08T03:21:50.979894+0000","last_peered":"2024-05-08T03:21:50.979894+0000","last_clean":"2024-05-08T03:21:50.979894+0000","last_became_active":"2024-05-08T03:20:37.788932+0000","last_became_peered":"2024-05-08T03:20:37.788932+0000","last_unstale":"2024-05-08T03:21:50.979894+0000","last_undegraded":"2024-05-08T03:21:50.979894+0000","last_fullsized":"2024-05-08T03:21:50.979894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:21.359546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767437,"stat_sum":{"num_bytes":10659372,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":7250,"num_read_kb":22534,"num_write":1012,"num_write_kb":9490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":2507487,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.883833+0000","last_change":"2024-05-08T03:20:47.820453+0000","last_active":"2024-05-08T03:21:50.883833+0000","last_peered":"2024-05-08T03:21:50.883833+0000","last_clean":"2024-05-08T03:21:50.883833+0000","last_became_active":"2024-05-08T03:20:47.820180+0000","last_became_peered":"2024-05-08T03:20:47.820180+0000","last_unstale":"2024-05-08T03:21:50.883833+0000","last_undegraded":"2024-05-08T03:21:50.883833+0000","last_fullsized":"2024-05-08T03:21:50.883833+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:24:33.427981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"150'443","reported_seq":4595,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.968493+0000","last_change":"2024-05-08T03:21:50.885862+0000","last_active":"2024-05-08T03:21:50.968493+0000","last_peered":"2024-05-08T03:21:50.968493+0000","last_clean":"2024-05-08T03:21:50.968493+0000","last_became_active":"2024-05-08T03:20:50.781137+0000","last_became_peered":"2024-05-08T03:20:50.781137+0000","last_unstale":"2024-05-08T03:21:50.968493+0000","last_undegraded":"2024-05-08T03:21:50.968493+0000","last_fullsized":"2024-05-08T03:21:50.968493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:09.446314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001884312,"stat_sum":{"num_bytes":5186877,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":24,"num_read":4090,"num_read_kb":14197,"num_write":428,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.6","version":"0'0","reported_seq":120,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.891546+0000","last_change":"2024-05-08T03:20:47.821931+0000","last_active":"2024-05-08T03:21:50.891546+0000","last_peered":"2024-05-08T03:21:50.891546+0000","last_clean":"2024-05-08T03:21:50.891546+0000","last_became_active":"2024-05-08T03:20:47.821642+0000","last_became_peered":"2024-05-08T03:20:47.821642+0000","last_unstale":"2024-05-08T03:21:50.891546+0000","last_undegraded":"2024-05-08T03:21:50.891546+0000","last_fullsized":"2024-05-08T03:21:50.891546+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:42.419280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"150'831","reported_seq":5160,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.001827+0000","last_change":"2024-05-08T03:21:50.887920+0000","last_active":"2024-05-08T03:21:51.001827+0000","last_peered":"2024-05-08T03:21:51.001827+0000","last_clean":"2024-05-08T03:21:51.001827+0000","last_became_active":"2024-05-08T03:19:20.774058+0000","last_became_peered":"2024-05-08T03:19:20.774058+0000","last_unstale":"2024-05-08T03:21:51.001827+0000","last_undegraded":"2024-05-08T03:21:51.001827+0000","last_fullsized":"2024-05-08T03:21:51.001827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:36.276321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023872780000000001,"stat_sum":{"num_bytes":21050562,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":25,"num_read":7337,"num_read_kb":34317,"num_write":1001,"num_write_kb":18524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.5","version":"0'0","reported_seq":146,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.892307+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:21:50.892307+0000","last_peered":"2024-05-08T03:21:50.892307+0000","last_clean":"2024-05-08T03:21:50.892307+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:21:50.892307+0000","last_undegraded":"2024-05-08T03:21:50.892307+0000","last_fullsized":"2024-05-08T03:21:50.892307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"150'314","reported_seq":5548,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.977052+0000","last_change":"2024-05-08T03:21:50.892941+0000","last_active":"2024-05-08T03:21:50.977052+0000","last_peered":"2024-05-08T03:21:50.977052+0000","last_clean":"2024-05-08T03:21:50.977052+0000","last_became_active":"2024-05-08T03:20:50.783297+0000","last_became_peered":"2024-05-08T03:20:50.783297+0000","last_unstale":"2024-05-08T03:21:50.977052+0000","last_undegraded":"2024-05-08T03:21:50.977052+0000","last_fullsized":"2024-05-08T03:21:50.977052+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:30.203547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011371510000000001,"stat_sum":{"num_bytes":2580846,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":22,"num_read":4508,"num_read_kb":31383,"num_write":326,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":13300321,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":148,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.900884+0000","last_change":"2024-05-08T03:21:20.923976+0000","last_active":"2024-05-08T03:21:50.900884+0000","last_peered":"2024-05-08T03:21:50.900884+0000","last_clean":"2024-05-08T03:21:50.900884+0000","last_became_active":"2024-05-08T03:21:20.923726+0000","last_became_peered":"2024-05-08T03:21:20.923726+0000","last_unstale":"2024-05-08T03:21:50.900884+0000","last_undegraded":"2024-05-08T03:21:50.900884+0000","last_fullsized":"2024-05-08T03:21:50.900884+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:52.436753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"150'999","reported_seq":4840,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.005670+0000","last_change":"2024-05-08T03:21:50.886189+0000","last_active":"2024-05-08T03:21:51.005670+0000","last_peered":"2024-05-08T03:21:51.005670+0000","last_clean":"2024-05-08T03:21:51.005670+0000","last_became_active":"2024-05-08T03:20:38.806015+0000","last_became_peered":"2024-05-08T03:20:38.806015+0000","last_unstale":"2024-05-08T03:21:51.005670+0000","last_undegraded":"2024-05-08T03:21:51.005670+0000","last_fullsized":"2024-05-08T03:21:51.005670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:08.292950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001818395,"stat_sum":{"num_bytes":24256806,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":25,"num_read":6090,"num_read_kb":34898,"num_write":1421,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":44040868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.1","version":"150'342","reported_seq":4119,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.979902+0000","last_change":"2024-05-08T03:21:50.892671+0000","last_active":"2024-05-08T03:21:50.979902+0000","last_peered":"2024-05-08T03:21:50.979902+0000","last_clean":"2024-05-08T03:21:50.979902+0000","last_became_active":"2024-05-08T03:20:50.783285+0000","last_became_peered":"2024-05-08T03:20:50.783285+0000","last_unstale":"2024-05-08T03:21:50.979902+0000","last_undegraded":"2024-05-08T03:21:50.979902+0000","last_fullsized":"2024-05-08T03:21:50.979902+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:42.546694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093165200000000004,"stat_sum":{"num_bytes":6039276,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":12,"num_read":2974,"num_read_kb":19727,"num_write":360,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":14979418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"2.0","version":"18'117","reported_seq":424,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.885839+0000","last_change":"2024-05-08T03:20:50.784319+0000","last_active":"2024-05-08T03:21:50.885839+0000","last_peered":"2024-05-08T03:21:50.885839+0000","last_clean":"2024-05-08T03:21:50.885839+0000","last_became_active":"2024-05-08T03:20:50.783795+0000","last_became_peered":"2024-05-08T03:20:50.783795+0000","last_unstale":"2024-05-08T03:21:50.885839+0000","last_undegraded":"2024-05-08T03:21:50.885839+0000","last_fullsized":"2024-05-08T03:21:50.885839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:44:15.151424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.885774+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:21:50.885774+0000","last_peered":"2024-05-08T03:21:50.885774+0000","last_clean":"2024-05-08T03:21:50.885774+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:21:50.885774+0000","last_undegraded":"2024-05-08T03:21:50.885774+0000","last_fullsized":"2024-05-08T03:21:50.885774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"150'477","reported_seq":7126,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.975500+0000","last_change":"2024-05-08T03:21:50.888494+0000","last_active":"2024-05-08T03:21:50.975500+0000","last_peered":"2024-05-08T03:21:50.975500+0000","last_clean":"2024-05-08T03:21:50.975500+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:21:50.975500+0000","last_undegraded":"2024-05-08T03:21:50.975500+0000","last_fullsized":"2024-05-08T03:21:50.975500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:43:23.464437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028669939999999999,"stat_sum":{"num_bytes":13788353,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":7208,"num_read_kb":29295,"num_write":475,"num_write_kb":32552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.892609+0000","last_change":"2024-05-08T03:20:34.779506+0000","last_active":"2024-05-08T03:21:50.892609+0000","last_peered":"2024-05-08T03:21:50.892609+0000","last_clean":"2024-05-08T03:21:50.892609+0000","last_became_active":"2024-05-08T03:20:34.779328+0000","last_became_peered":"2024-05-08T03:20:34.779328+0000","last_unstale":"2024-05-08T03:21:50.892609+0000","last_undegraded":"2024-05-08T03:21:50.892609+0000","last_fullsized":"2024-05-08T03:21:50.892609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:33.865049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"150'914","reported_seq":8827,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.007225+0000","last_change":"2024-05-08T03:21:50.893290+0000","last_active":"2024-05-08T03:21:51.007225+0000","last_peered":"2024-05-08T03:21:51.007225+0000","last_clean":"2024-05-08T03:21:51.007225+0000","last_became_active":"2024-05-08T03:20:40.780249+0000","last_became_peered":"2024-05-08T03:20:40.780249+0000","last_unstale":"2024-05-08T03:21:51.007225+0000","last_undegraded":"2024-05-08T03:21:51.007225+0000","last_fullsized":"2024-05-08T03:21:51.007225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:16.444446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013847180000000001,"stat_sum":{"num_bytes":22969451,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":11432,"num_read_kb":55666,"num_write":1116,"num_write_kb":28085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.1","version":"0'0","reported_seq":279,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.883629+0000","last_change":"2024-05-08T03:20:34.779020+0000","last_active":"2024-05-08T03:21:50.883629+0000","last_peered":"2024-05-08T03:21:50.883629+0000","last_clean":"2024-05-08T03:21:50.883629+0000","last_became_active":"2024-05-08T03:20:34.778668+0000","last_became_peered":"2024-05-08T03:20:34.778668+0000","last_unstale":"2024-05-08T03:21:50.883629+0000","last_undegraded":"2024-05-08T03:21:50.883629+0000","last_fullsized":"2024-05-08T03:21:50.883629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:52:48.100988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"150'883","reported_seq":9904,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.957631+0000","last_change":"2024-05-08T03:21:50.894409+0000","last_active":"2024-05-08T03:21:50.957631+0000","last_peered":"2024-05-08T03:21:50.957631+0000","last_clean":"2024-05-08T03:21:50.957631+0000","last_became_active":"2024-05-08T03:20:52.799061+0000","last_became_peered":"2024-05-08T03:20:52.799061+0000","last_unstale":"2024-05-08T03:21:50.957631+0000","last_undegraded":"2024-05-08T03:21:50.957631+0000","last_fullsized":"2024-05-08T03:21:50.957631+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:16:12.956042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020585540000000002,"stat_sum":{"num_bytes":20652985,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":11196,"num_read_kb":56202,"num_write":1149,"num_write_kb":16998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.884962+0000","last_change":"2024-05-08T03:20:03.022289+0000","last_active":"2024-05-08T03:21:50.884962+0000","last_peered":"2024-05-08T03:21:50.884962+0000","last_clean":"2024-05-08T03:21:50.884962+0000","last_became_active":"2024-05-08T03:20:03.022003+0000","last_became_peered":"2024-05-08T03:20:03.022003+0000","last_unstale":"2024-05-08T03:21:50.884962+0000","last_undegraded":"2024-05-08T03:21:50.884962+0000","last_fullsized":"2024-05-08T03:21:50.884962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:33:37.476106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"150'515","reported_seq":5006,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.037772+0000","last_change":"2024-05-08T03:21:50.887340+0000","last_active":"2024-05-08T03:21:51.037772+0000","last_peered":"2024-05-08T03:21:51.037772+0000","last_clean":"2024-05-08T03:21:51.037772+0000","last_became_active":"2024-05-08T03:20:51.780777+0000","last_became_peered":"2024-05-08T03:20:51.780777+0000","last_unstale":"2024-05-08T03:21:51.037772+0000","last_undegraded":"2024-05-08T03:21:51.037772+0000","last_fullsized":"2024-05-08T03:21:51.037772+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:10.487958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019784270000000001,"stat_sum":{"num_bytes":4014096,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":6851,"num_read_kb":17680,"num_write":484,"num_write_kb":9915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5273218,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.9","version":"150'524","reported_seq":3862,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.971809+0000","last_change":"2024-05-08T03:21:50.893652+0000","last_active":"2024-05-08T03:21:50.971809+0000","last_peered":"2024-05-08T03:21:50.971809+0000","last_clean":"2024-05-08T03:21:50.971809+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:21:50.971809+0000","last_undegraded":"2024-05-08T03:21:50.971809+0000","last_fullsized":"2024-05-08T03:21:50.971809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:41:24.851529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602773,"stat_sum":{"num_bytes":75981253,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":4257,"num_read_kb":26399,"num_write":658,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.a","version":"150'586","reported_seq":2231,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.971914+0000","last_change":"2024-05-08T03:21:50.894612+0000","last_active":"2024-05-08T03:21:50.971914+0000","last_peered":"2024-05-08T03:21:50.971914+0000","last_clean":"2024-05-08T03:21:50.971914+0000","last_became_active":"2024-05-08T03:20:39.775306+0000","last_became_peered":"2024-05-08T03:20:39.775306+0000","last_unstale":"2024-05-08T03:21:50.971914+0000","last_undegraded":"2024-05-08T03:21:50.971914+0000","last_fullsized":"2024-05-08T03:21:50.971914+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:59.166697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838138,"stat_sum":{"num_bytes":22123723,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":15,"num_read":3188,"num_read_kb":10391,"num_write":627,"num_write_kb":5635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.b","version":"150'437","reported_seq":7536,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.982837+0000","last_change":"2024-05-08T03:21:50.888284+0000","last_active":"2024-05-08T03:21:50.982837+0000","last_peered":"2024-05-08T03:21:50.982837+0000","last_clean":"2024-05-08T03:21:50.982837+0000","last_became_active":"2024-05-08T03:21:20.923471+0000","last_became_peered":"2024-05-08T03:21:20.923471+0000","last_unstale":"2024-05-08T03:21:50.982837+0000","last_undegraded":"2024-05-08T03:21:50.982837+0000","last_fullsized":"2024-05-08T03:21:50.982837+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:08:15.803291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002813692,"stat_sum":{"num_bytes":38349493,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":6764,"num_read_kb":58890,"num_write":460,"num_write_kb":18516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":35463562,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.c","version":"150'784","reported_seq":5078,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.020186+0000","last_change":"2024-05-08T03:21:50.886846+0000","last_active":"2024-05-08T03:21:51.020186+0000","last_peered":"2024-05-08T03:21:51.020186+0000","last_clean":"2024-05-08T03:21:51.020186+0000","last_became_active":"2024-05-08T03:20:52.802662+0000","last_became_peered":"2024-05-08T03:20:52.802662+0000","last_unstale":"2024-05-08T03:21:51.020186+0000","last_undegraded":"2024-05-08T03:21:51.020186+0000","last_fullsized":"2024-05-08T03:21:51.020186+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:41.171871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001543544,"stat_sum":{"num_bytes":76890608,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":30,"num_read":6871,"num_read_kb":23505,"num_write":1041,"num_write_kb":23621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.d","version":"150'1046","reported_seq":5455,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.050837+0000","last_change":"2024-05-08T03:21:50.892596+0000","last_active":"2024-05-08T03:21:51.050837+0000","last_peered":"2024-05-08T03:21:51.050837+0000","last_clean":"2024-05-08T03:21:51.050837+0000","last_became_active":"2024-05-08T03:20:51.783022+0000","last_became_peered":"2024-05-08T03:20:51.783022+0000","last_unstale":"2024-05-08T03:21:51.050837+0000","last_undegraded":"2024-05-08T03:21:51.050837+0000","last_fullsized":"2024-05-08T03:21:51.050837+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:12:33.470181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095591100000000004,"stat_sum":{"num_bytes":54706144,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":9206,"num_read_kb":75054,"num_write":1337,"num_write_kb":49365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":48212116,"num_keys_recovered":59,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.e","version":"150'630","reported_seq":4597,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.988199+0000","last_change":"2024-05-08T03:21:50.892978+0000","last_active":"2024-05-08T03:21:50.988199+0000","last_peered":"2024-05-08T03:21:50.988199+0000","last_clean":"2024-05-08T03:21:50.988199+0000","last_became_active":"2024-05-08T03:20:40.779118+0000","last_became_peered":"2024-05-08T03:20:40.779118+0000","last_unstale":"2024-05-08T03:21:50.988199+0000","last_undegraded":"2024-05-08T03:21:50.988199+0000","last_fullsized":"2024-05-08T03:21:50.988199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:41.035214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012124620000000001,"stat_sum":{"num_bytes":48175019,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":4631,"num_read_kb":56936,"num_write":811,"num_write_kb":30706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":51071767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.f","version":"150'600","reported_seq":4051,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.988172+0000","last_change":"2024-05-08T03:21:50.893514+0000","last_active":"2024-05-08T03:21:50.988172+0000","last_peered":"2024-05-08T03:21:50.988172+0000","last_clean":"2024-05-08T03:21:50.988172+0000","last_became_active":"2024-05-08T03:20:38.805719+0000","last_became_peered":"2024-05-08T03:20:38.805719+0000","last_unstale":"2024-05-08T03:21:50.988172+0000","last_undegraded":"2024-05-08T03:21:50.988172+0000","last_fullsized":"2024-05-08T03:21:50.988172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:19.138013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015288210000000001,"stat_sum":{"num_bytes":10819069,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":22,"num_read":4007,"num_read_kb":15823,"num_write":769,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":10326037,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.10","version":"150'505","reported_seq":4983,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.957188+0000","last_change":"2024-05-08T03:21:50.886471+0000","last_active":"2024-05-08T03:21:50.957188+0000","last_peered":"2024-05-08T03:21:50.957188+0000","last_clean":"2024-05-08T03:21:50.957188+0000","last_became_active":"2024-05-08T03:20:51.782569+0000","last_became_peered":"2024-05-08T03:20:51.782569+0000","last_unstale":"2024-05-08T03:21:50.957188+0000","last_undegraded":"2024-05-08T03:21:50.957188+0000","last_fullsized":"2024-05-08T03:21:50.957188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:33.816535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019753069999999999,"stat_sum":{"num_bytes":55478607,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":25,"num_read":5994,"num_read_kb":38231,"num_write":474,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.11","version":"150'520","reported_seq":6080,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.031771+0000","last_change":"2024-05-08T03:21:50.893239+0000","last_active":"2024-05-08T03:21:51.031771+0000","last_peered":"2024-05-08T03:21:51.031771+0000","last_clean":"2024-05-08T03:21:51.031771+0000","last_became_active":"2024-05-08T03:20:38.805730+0000","last_became_peered":"2024-05-08T03:20:38.805730+0000","last_unstale":"2024-05-08T03:21:51.031771+0000","last_undegraded":"2024-05-08T03:21:51.031771+0000","last_fullsized":"2024-05-08T03:21:51.031771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:50.629498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013342300000000001,"stat_sum":{"num_bytes":49500365,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":19,"num_read":7107,"num_read_kb":66081,"num_write":488,"num_write_kb":56641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":60512583,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.12","version":"150'388","reported_seq":2930,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:50.955824+0000","last_change":"2024-05-08T03:21:50.887930+0000","last_active":"2024-05-08T03:21:50.955824+0000","last_peered":"2024-05-08T03:21:50.955824+0000","last_clean":"2024-05-08T03:21:50.955824+0000","last_became_active":"2024-05-08T03:20:47.824316+0000","last_became_peered":"2024-05-08T03:20:47.824316+0000","last_unstale":"2024-05-08T03:21:50.955824+0000","last_undegraded":"2024-05-08T03:21:50.955824+0000","last_fullsized":"2024-05-08T03:21:50.955824+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:15.460268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002405197,"stat_sum":{"num_bytes":34208895,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":4162,"num_read_kb":29959,"num_write":405,"num_write_kb":31580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":5098174,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"5"}]},{"pgid":"3.13","version":"150'407","reported_seq":5412,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-05-08T03:21:51.053716+0000","last_change":"2024-05-08T03:21:50.884586+0000","last_active":"2024-05-08T03:21:51.053716+0000","last_peered":"2024-05-08T03:21:51.053716+0000","last_clean":"2024-05-08T03:21:51.053716+0000","last_became_active":"2024-05-08T03:21:20.922351+0000","last_became_peered":"2024-05-08T03:21:20.922351+0000","last_unstale":"2024-05-08T03:21:51.053716+0000","last_undegraded":"2024-05-08T03:21:51.053716+0000","last_fullsized":"2024-05-08T03:21:51.053716+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:58:09.499349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073671100000000003,"stat_sum":{"num_bytes":18959108,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":28,"num_read":4936,"num_read_kb":49256,"num_write":530,"num_write_kb":27657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":40436240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"15","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":996510136,"num_objects":1106,"num_object_clones":148,"num_object_copies":2212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1106,"num_whiteouts":706,"num_read":194223,"num_read_kb":1188424,"num_write":22888,"num_write_kb":748317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4136,"num_bytes_recovered":610881052,"num_keys_recovered":712,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":818020352,"data_stored":1208837884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19127,"ondisk_log_size":19127,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":60129542179,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":178464,"kb_used_data":110928,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94193376,"statfs":{"total":96636764160,"available":96454017024,"internally_reserved":0,"allocated":113590272,"data_stored":168629388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":0,"seq":455266533387,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":270512,"kb_used_data":207912,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94101328,"statfs":{"total":96636764160,"available":96359759872,"internally_reserved":0,"allocated":212901888,"data_stored":306439346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":60129542180,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157208,"kb_used_data":103820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94214632,"statfs":{"total":96636764160,"available":96475783168,"internally_reserved":0,"allocated":106311680,"data_stored":161596145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223920,"kb_used_data":152484,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94147920,"statfs":{"total":96636764160,"available":96407470080,"internally_reserved":0,"allocated":156143616,"data_stored":228876775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542180,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22142976,"data_stored":29440943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152809472,"data_stored":226498013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":103432192,"data_stored":159660824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":135417856,"data_stored":199525057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":211103744,"data_stored":305566616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61374464,"data_stored":83949758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":109654016,"data_stored":165648979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22085632,"data_stored":38547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:21:53.100 INFO:tasks.thrashosds.thrasher:Setting 3.14 to [2, 0, 1, 7] 2024-05-08T03:21:53.100 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.14', '2', '0', '1', '7'] 2024-05-08T03:21:53.100 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.14 2 0 1 7 2024-05-08T03:21:53.118 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:53.117+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:53.143 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.143 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.153 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:53.219 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:53.219+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:53.261 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.261 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.270 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:53.320 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:53.319+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:53.377 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.377 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.386 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:53.429 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:53.428+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:53.492 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.492 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.501 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:53.522 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:53.521+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:53.627 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:53.626+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:53.646 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.646 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.656 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:53.703 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:53.728 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:53.727+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:53.774 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:53.774 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:53.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:53.825 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:53.825 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:53.828 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:53.827+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:53.842 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:53.842 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:53.859 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:53.859 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:53.875 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:53.875 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:53.892 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:53.893 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:53.909 INFO:teuthology.orchestra.run.smithi177.stderr:set 3.14 pg_upmap_items mapping to [2->0,1->7] 2024-05-08T03:21:53.916 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:53.916 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:53.934 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:53.934 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:53.934 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:53.934 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:53.934 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:53.936 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:53.936 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:53.937 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:53.935+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:53.944 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:53.955 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:53.955 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:54.030 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:54.029+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:54.055 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.280207085999997, 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.056 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:21:54.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.265583880999998, 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:21:54.061 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:21:54.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:21:54.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:21:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.264774602000003, 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.067 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:54.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.263835325000002, 2024-05-08T03:21:54.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.262202254999998, 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:54.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:21:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:21:54.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:21:54.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:21:54.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.085 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.260763759, 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:21:54.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:21:54.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:21:54.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.260743871000003, 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.090 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.260719952000002, 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.095 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.259748772000002, 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.259737162999997, 2024-05-08T03:21:54.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.259727445000003, 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.109 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:21:54.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.113 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.259716982999997, 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.114 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:54.118 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.257140036999999, 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:21:54.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.257109790000001, 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:21:54.123 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:21:54.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.257082773, 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.128 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.257063780000003, 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.133 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.135 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.136 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.257033720999999, 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.138 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.140 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.141 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.255249257000003, 2024-05-08T03:21:54.142 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.143 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.144 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.146 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.255237434000001, 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.147 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.151 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 63.255225576999997, 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.152 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:54.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:21:54.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:21:54.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.156 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:54.158 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:54.158 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:54.156+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:54.259 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:54.258+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:54.262 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:54.262 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:54.262 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:54.262 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:54.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:54.359 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:54.358+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:54.416 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:54.417 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:54.417 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:54.417 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:54.417 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:54.425 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:54.460 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:54.459+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:54.531 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.320455299, 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:21:54.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40072498800001, 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40069327099999, 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.542 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:21:54.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:21:54.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:21:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40066816699999, 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:21:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40064336200001, 2024-05-08T03:21:54.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:54.554 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40061481800001, 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.558 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40059229600001, 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:21:54.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:21:54.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:21:54.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.40056195599999, 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.564 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:21:54.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.569 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 137.400540775, 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.570 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 123.587898839, 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.574 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:21:54.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.39922411800001, 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.581 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:21:54.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:21:54.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:21:54.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:21:54.586 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.37702077500001, 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:21:54.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:21:54.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:21:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.37532465699999, 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:21:54.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.595 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.35561986899999, 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.596 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:21:54.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:21:54.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.355595909, 2024-05-08T03:21:54.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:21:54.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:21:54.608 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.315004364, 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:21:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:21:54.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:21:54.612 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 114.302230467, 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:21:54.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:21:54.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:21:54.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 95.041491532999999, 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 39.683187134000001, 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.624 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 39.681568396999999, 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:54.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:54.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:54.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:54.629 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:54.631 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:54.632 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:54.629+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:54.732 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:54.731+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:54.737 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:54.737 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:54.737 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:54.737 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:54.746 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:54.833 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:54.831+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:54.863 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:54.863 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:54.863 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:54.863 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:54.863 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:54.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:54.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:21:54.964 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:54.962+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 143.14485308299999, 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:21:55.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:21:55.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.027 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5004 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80291189100001, 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031113175999999999, 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5004 2024-05-08T03:21:55.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216074+0000", 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7099999999999998e-07 2024-05-08T03:21:55.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216075+0000", 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2589999999999999e-06 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216078+0000", 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0800000000000002e-06 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216081+0000", 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3209999999999999e-06 2024-05-08T03:21:55.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247092+0000", 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031011555 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247133+0000", 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0717999999999999e-05 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247187+0000", 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3971999999999998e-05 2024-05-08T03:21:55.031 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5005 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3010560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.802896492, 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031293209000000002, 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.032 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5005 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.033 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216089+0000", 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216095+0000", 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9020000000000004e-06 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216102+0000", 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1360000000000002e-06 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216106+0000", 2024-05-08T03:21:55.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9049999999999996e-06 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247219+0000", 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031112320999999998 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247291+0000", 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1976000000000003e-05 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247382+0000", 2024-05-08T03:21:55.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1558999999999995e-05 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5006 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3026944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80286484999999, 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031387353999999999, 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.037 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5006 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216121+0000", 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5699999999999998e-07 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216123+0000", 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9989999999999998e-06 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216128+0000", 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.34e-06 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216131+0000", 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.0759999999999999e-06 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247416+0000", 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031285885999999999 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247455+0000", 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8742000000000003e-05 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247508+0000", 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2853999999999999e-05 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5007 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3018752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80285151800001, 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031568763999999999, 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.041 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.042 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.042 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5007 2024-05-08T03:21:55.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216134+0000", 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8000000000000002e-07 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216135+0000", 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216138+0000", 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8899999999999999e-06 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216141+0000", 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7389999999999999e-06 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247540+0000", 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031399023999999998 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247611+0000", 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1130000000000005e-05 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247703+0000", 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1611000000000001e-05 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5008 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3035136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80284172399999, 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031677899000000002, 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.048 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5008 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:55.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216144+0000", 2024-05-08T03:21:55.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9299999999999999e-07 2024-05-08T03:21:55.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216145+0000", 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0899999999999999e-06 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216148+0000", 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7879999999999998e-06 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216151+0000", 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4839999999999998e-06 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247729+0000", 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031578389999999998 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247767+0000", 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8302e-05 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247822+0000", 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4552000000000002e-05 2024-05-08T03:21:55.053 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.802832311, 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.054 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80282278999999, 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.058 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:21:55.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:21:55.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80281344900001, 2024-05-08T03:21:55.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80280423400001, 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.070 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80279487199999, 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.071 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:21:55.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:21:55.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80278544000001, 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.075 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.802776024, 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.079 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:55.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80276700600001, 2024-05-08T03:21:55.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:21:55.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.802757963, 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.091 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80274567000001, 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.092 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:21:55.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:21:55.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:21:55.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80273640300001, 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.096 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80272254499999, 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.100 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.80271325199999, 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.104 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:12442 3.9 e33/28 3:97a59041:::rbd_data.1182ca0d3624.0000000000000000:head v 33'21, mlcod=32'19)", 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.80969911899999, 2024-05-08T03:21:55.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031642361000000001, 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209286+0000", 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.209287+0000", 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8699999999999995e-07 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219629+0000", 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.010341703000000001 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219644+0000", 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5231999999999999e-05 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219650+0000", 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5560000000000004e-06 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219720+0000", 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9776000000000004e-05 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219756+0000", 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5689000000000003e-05 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240880+0000", 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021124229000000001 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.240929+0000", 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8689000000000001e-05 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:21:55.112 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:55.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:55.114 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:55.112+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:55.215 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:55.214+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:55.220 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:55.220 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:55.220 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:55.220 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:55.229 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:55.316 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:55.315+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:55.373 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:55.373 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:55.373 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:55.373 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:55.374 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:55.382 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:55.416 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:55.415+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:55.517 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:55.516+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:55.527 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:55.527 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:55.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:55.527 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:55.527 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:55.536 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:55.618 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:55.617+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:55.645 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:55.645 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:55.646 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:55.646 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:55.654 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:55.738 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:55.737+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:55.799 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:55.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:55.800 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:55.808 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:55.820 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:55.819+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:55.905 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:21:55.920 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:55.919+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:55.964 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.762222726, 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.965 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:55.966 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:55.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.967 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.967 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.967 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:55.967 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:55.967 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:55.968 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:55.969 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.76221327600001, 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.970 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.971 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.972 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.973 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.76220054000001, 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.974 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.975 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.976 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.977 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.76218996399999, 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:55.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:55.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:55.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.76218057700001, 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:55.986 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.762171397, 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.987 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:55.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:55.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:55.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53134636199999, 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53131412499999, 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:55.995 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:55.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:55.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:55.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53130079499999, 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:55.999 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:56.003 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53128582100001, 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:56.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:56.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:56.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53127451200001, 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.008 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:56.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:56.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.012 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53125684299999, 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.53124164499999, 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.017 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 147.531227551, 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.021 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.025 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 139.75627357799999, 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.026 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:56.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.031 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 138.83337977599999, 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:56.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 138.83336628800001, 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.036 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 138.83333730699999, 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 138.83331070099999, 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.044 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:56.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 138.40056243999999, 2024-05-08T03:21:56.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:56.054 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:56.054 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:56.054 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:21:56.056 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:56.056 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:56.054+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:56.156 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:56.155+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:56.206 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:56.206 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:56.206 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:56.206 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:56.219 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:56.227 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:56.228 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:56.246 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:56.246 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:56.257 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:56.256+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:56.264 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:56.264 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:56.281 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:56.282 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:56.299 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:56.299 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:56.317 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:56.317 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:56.328 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:56.328 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:56.333 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:56.334 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:56.337 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:56.352 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:56.352 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:56.363 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:56.361+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:56.457 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:56.457 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:56.459 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:56.458+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:56.466 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:21:56.560 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:56.559+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:56.571 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:56.571 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:56.580 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:21:56.663 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:56.662+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:56.724 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:56.724 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:56.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:21:56.762 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:56.762+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:56.863 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:56.862+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:56.878 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:56.878 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:56.887 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:21:56.971 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:56.970+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:57.041 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.041 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.050 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:21:57.065 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:57.064+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:57.165 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:57.165+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:57.198 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.198 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.207 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:21:57.266 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:57.265+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:57.312 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.312 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.321 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:21:57.367 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:57.367+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:57.430 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.430 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:21:57.468 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:57.467+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:57.570 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:57.568+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:57.583 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.583 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.592 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:21:57.671 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:57.670+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:57.697 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.698 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:21:57.791 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:57.790+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:57.852 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:57.853 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:57.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:21:57.903 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:21:57.902+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:21:57.925 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:21:57.973 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:57.972+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:58.023 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:58.024 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:58.034 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:21:58.077 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:58.076+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:58.138 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:58.138 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:58.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:21:58.176 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:58.174+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:58.277 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:58.276+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:58.292 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:58.293 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:58.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:21:58.364 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:21:58.385 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:58.384+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:58.448 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:58.448 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:58.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:21:58.478 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:21:58.477+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:21:58.486 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:21:58.486 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:21:58.504 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:21:58.504 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:21:58.520 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:21:58.520 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:21:58.538 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:21:58.538 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:21:58.555 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:21:58.555 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:21:58.572 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:21:58.573 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:21:58.579 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:58.578+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:58.589 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:21:58.590 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:21:58.604 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:21:58.604 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:21:58.606 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:21:58.607 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:21:58.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:21:58.698 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:21:58.697+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:21:58.759 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:58.759 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:58.759 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:21:58.759 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:58.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:21:58.781 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:58.780+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:58.882 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:21:58.880+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:21:58.913 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:58.913 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:21:58.913 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:21:58.913 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:21:58.913 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:58.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:21:58.925 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4] out_osds: [7, 0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:21:58.925 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:21:58.925 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-05-08T03:21:58.926 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-05-08T03:21:58.930 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:21:58.982 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:21:58.981+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:21:59.037 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:21:59.038 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:21:59.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:21:59.038 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:21:59.038 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:21:59.047 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:21:59.090 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:59.089+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:59.156 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:59.156 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:59.156 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:59.156 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:59.165 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:21:59.206 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:21:59.204+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:21:59.246 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:21:59.244+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e158 definitely_dead 0 2024-05-08T03:21:59.286 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:59.284+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:59.309 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:59.309 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:59.309 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:59.310 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:59.310 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:59.319 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:21:59.387 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:59.386+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:59.465 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:59.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.263485645, 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:21:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:21:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.469 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.26347619500001, 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.470 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.26346345900001, 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.474 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.475 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.26345288300001, 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.478 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.26344349600001, 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.482 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.263434316, 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03260928099999, 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.491 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03257704399999, 2024-05-08T03:21:59.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.497 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.498 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03256371399999, 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:21:59.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:21:59.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.502 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03254874000001, 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.504 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03253743100001, 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.509 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.510 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03251976199999, 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.511 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:21:59.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:21:59.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:21:59.514 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03250456399999, 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.515 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 151.03249047, 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.519 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:21:59.523 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 143.25753649699999, 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:59.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:59.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 142.33464269500001, 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.529 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 142.33462920700001, 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:21:59.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 142.33460022599999, 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.537 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:59.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:21:59.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:21:59.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 142.33457361999999, 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:21:59.541 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:21:59.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:21:59.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.545 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 141.90182535899999, 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:59.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.551 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:59.553 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:21:59.553 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:59.551+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:59.653 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:59.652+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:59.699 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:59.699 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:59.699 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:21:59.699 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:59.708 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:21:59.754 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:21:59.754+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:21:59.782 INFO:teuthology.orchestra.run.smithi177.stderr:marked in osd.7. 2024-05-08T03:21:59.803 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-05-08T03:21:59.823 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:59.824 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:21:59.824 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:21:59.824 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:21:59.824 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:21:59.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:21:59.874 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:59.872+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:59.956 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:21:59.955+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:21:59.981 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 130.04669471700001, 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.982 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:21:59.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.186518187000004, 2024-05-08T03:21:59.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.992 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:21:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:21:59.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:21:59.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.996 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.186492161999993, 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:21:59.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:21:59.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:22:00.001 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.186450147000002, 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:22:00.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:22:00.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:22:00.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.185467470999995, 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.006 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:22:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:22:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.185431554999994, 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:22:00.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:22:00.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.015 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.185406513000004, 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.016 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:22:00.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:22:00.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.182996592999999, 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:22:00.022 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:22:00.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.028 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.182972298999999, 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.029 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:22:00.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:22:00.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:22:00.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.182899182, 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.034 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.038 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.182870949000005, 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:22:00.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.182840201999994, 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.044 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:22:00.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:22:00.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:22:00.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.182802843000005, 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:22:00.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:22:00.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181201111999997, 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:22:00.054 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:22:00.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:22:00.060 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181147091, 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:22:00.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:22:00.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.066 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181114385000001, 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:22:00.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:22:00.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181089993000001, 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:22:00.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:22:00.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:22:00.076 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181070571999996, 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:22:00.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:22:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181059293000004, 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.082 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.086 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 69.181039599000002, 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.087 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:00.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.092 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:00.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:22:00.095 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:00.093+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:00.195 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:00.193+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:00.240 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:00.241 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:00.241 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:00.241 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:00.249 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:22:00.334 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:00.332+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:00.395 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:00.395 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:00.395 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:00.395 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:00.396 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:00.397 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:00.395+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:00.404 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:22:00.497 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:00.496+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:00.549 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:00.549 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:22:00.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:22:00.549 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.44701456600001, 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.550 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.552 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:22:00.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.555 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.445294096, 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:22:00.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:22:00.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.558 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:22:00.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.445260655, 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.560 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:22:00.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:22:00.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.564 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.443834916, 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.443742302, 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.569 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:00.573 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.44323162800001, 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.574 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:22:00.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:22:00.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:22:00.577 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.44320412, 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.578 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.583 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.442797689, 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:22:00.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:22:00.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:22:00.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:22:00.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.442342311, 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.441814064, 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.44134156600001, 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.44126343400001, 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:22:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:22:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:22:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:22:00.606 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.441188012, 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:22:00.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:22:00.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:22:00.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.441110849, 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.441033326, 2024-05-08T03:22:00.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.619 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.441006414, 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:22:00.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:22:00.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:22:00.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.440985204, 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.624 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:22:00.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:22:00.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:22:00.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.628 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.440965424, 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:00.629 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:22:00.630 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:22:00.631 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:22:00.632 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.440410898, 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.633 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.634 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.635 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.636 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.637 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 112.440391886, 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:00.638 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:00.639 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:00.640 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:00.641 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:00.642 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:00.643 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:22:00.644 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:22:00.645 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:00.642+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:00.744 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:00.743+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:00.799 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:00.799 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:00.799 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:00.799 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:00.809 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:22:00.814 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:00.815 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:00.833 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:00.845 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:00.844+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:00.850 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:00.850 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:00.867 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:00.867 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:00.884 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:00.885 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:00.902 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:00.903 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:00.916 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:00.916 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:00.920 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:00.920 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:00.925 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:22:00.938 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:00.938 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:00.946 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:00.945+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:00.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:22:01.045 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.045 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.046 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:01.046+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:01.055 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:22:01.148 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:01.146+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:01.171 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.172 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:22:01.249 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:01.247+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:01.286 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:22:01.350 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:01.349+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:01.400 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.400 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:22:01.451 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:01.449+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:01.551 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:01.550+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:01.555 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.555 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:22:01.652 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:01.651+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:01.668 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.669 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.677 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:22:01.753 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:01.752+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:01.784 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.784 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.792 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:22:01.793 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:22:01.854 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:01.853+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:01.937 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:01.937 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:01.946 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:22:01.954 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:01.953+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:02.055 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:02.054+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:02.075 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.075 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:22:02.156 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:02.155+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:02.208 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.208 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:22:02.259 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:02.258+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:02.357 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:02.356+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:02.375 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.376 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:22:02.458 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:02.457+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:02.499 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.499 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:22:02.559 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:02.558+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:02.626 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.627 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.636 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:22:02.678 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:02.677+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:02.745 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.745 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.754 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:22:02.761 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:02.760+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:02.861 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:02.860+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:02.873 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:02.874 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:02.884 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:22:02.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:22:02.962 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:02.961+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:03.042 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:03.043 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:03.052 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:22:03.062 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:03.062+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:03.094 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:03.095 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:03.113 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:03.113 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:03.130 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:03.130 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:03.147 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:03.147 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:03.163 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:03.163 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:03.163 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:03.163 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:03.164 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:03.162+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:03.164 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:03.164 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:03.172 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:22:03.181 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:03.182 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:03.199 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:03.200 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:03.216 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:03.217 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:03.264 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:03.262+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:03.300 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:03.300 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:03.300 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:03.300 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:03.300 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:03.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:22:03.366 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:03.364+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:03.419 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:20946 3.f 3:f139a7a9:::rbd_data.1182cef43756.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 133.48383306299999, 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0050859350000001, 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.420 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 20946 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.421 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932528+0000", 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6300000000000005e-07 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932531+0000", 2024-05-08T03:22:03.422 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8609999999999998e-06 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932540+0000", 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4800000000000001e-06 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932545+0000", 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1120000000000004e-06 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932599+0000", 2024-05-08T03:22:03.423 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4228000000000003e-05 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:49.932610+0000", 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0478e-05 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.928719+0000", 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99610917200000004 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for degraded object", 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.930189+0000", 2024-05-08T03:22:03.424 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001470332 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931702+0000", 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0015133810000000001 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931791+0000", 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9091999999999993e-05 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.931919+0000", 2024-05-08T03:22:03.425 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012711299999999999 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.933068+0000", 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0011494490000000001 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937468+0000", 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0043994719999999998 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937523+0000", 2024-05-08T03:22:03.426 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5856999999999997e-05 2024-05-08T03:22:03.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.427 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.427 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.427 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:50.937614+0000", 2024-05-08T03:22:03.427 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0345000000000003e-05 2024-05-08T03:22:03.427 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.429 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.430 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.430 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105426 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.430 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:22:03.430 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.623656533000002, 2024-05-08T03:22:03.430 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990395515, 2024-05-08T03:22:03.430 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105426 2024-05-08T03:22:03.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.432 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792704+0000", 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.433 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792705+0000", 2024-05-08T03:22:03.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8500000000000002e-07 2024-05-08T03:22:03.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.434 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.434 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.434 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792708+0000", 2024-05-08T03:22:03.434 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792716+0000", 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.435 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792722+0000", 2024-05-08T03:22:03.436 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-06 2024-05-08T03:22:03.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.436 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.436 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.436 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792894+0000", 2024-05-08T03:22:03.437 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000172139 2024-05-08T03:22:03.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.437 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.437 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.437 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792920+0000", 2024-05-08T03:22:03.438 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5986000000000002e-05 2024-05-08T03:22:03.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.438 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.438 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.438 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.782945+0000", 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99002446499999996 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783024+0000", 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.9146000000000004e-05 2024-05-08T03:22:03.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783100+0000", 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5798999999999997e-05 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.440 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.441 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.441 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.441 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105427 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.441 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:22:03.441 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.623630508000005, 2024-05-08T03:22:03.442 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99051268699999995, 2024-05-08T03:22:03.442 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.442 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.442 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.442 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.443 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.443 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105427 2024-05-08T03:22:03.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.443 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.443 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.443 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792730+0000", 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792731+0000", 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4700000000000002e-07 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.445 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792733+0000", 2024-05-08T03:22:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.367e-06 2024-05-08T03:22:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792739+0000", 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1850000000000001e-06 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792744+0000", 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7450000000000003e-06 2024-05-08T03:22:03.447 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792956+0000", 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000212396 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.448 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792961+0000", 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3429999999999998e-06 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783126+0000", 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99016521800000001 2024-05-08T03:22:03.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783192+0000", 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5686999999999996e-05 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.450 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783243+0000", 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1298999999999998e-05 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.451 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105428 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.623588493, 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99060789100000002, 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.452 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105428 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.453 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:22:03.454 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.454 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.454 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.454 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.454 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792772+0000", 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792773+0000", 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9699999999999996e-07 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.455 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792776+0000", 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4450000000000001e-06 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.456 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792784+0000", 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0260000000000007e-06 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792790+0000", 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5550000000000002e-06 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.457 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792981+0000", 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000190922 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.792987+0000", 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0209999999999996e-06 2024-05-08T03:22:03.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783266+0000", 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99027871700000003 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.459 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783323+0000", 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7473999999999999e-05 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.783380+0000", 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7234000000000001e-05 2024-05-08T03:22:03.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105441 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.622605816999993, 2024-05-08T03:22:03.461 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990448879, 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.462 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105441 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.463 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793755+0000", 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793756+0000", 2024-05-08T03:22:03.464 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7000000000000004e-07 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793760+0000", 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5469999999999998e-06 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.465 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793769+0000", 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9639999999999992e-06 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793776+0000", 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.669e-06 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.466 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793823+0000", 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7108999999999999e-05 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793835+0000", 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.219e-05 2024-05-08T03:22:03.467 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784036+0000", 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99020102799999998 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.468 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784128+0000", 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1725999999999996e-05 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784204+0000", 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.5876000000000003e-05 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.469 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105442 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.622569901000006, 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99052967000000003, 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.470 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105442 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.471 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793791+0000", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9599999999999999e-07 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793794+0000", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8399999999999999e-06 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793802+0000", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4730000000000003e-06 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.472 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793808+0000", 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9490000000000001e-06 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793878+0000", 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9985000000000002e-05 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793886+0000", 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.6789999999999994e-06 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.473 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784233+0000", 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99034691399999997 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784279+0000", 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6084000000000001e-05 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784321+0000", 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.125e-05 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.474 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105443 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.622544859000001, 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99061739999999998, 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105443 2024-05-08T03:22:03.475 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.476 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793816+0000", 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2300000000000002e-07 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793819+0000", 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3580000000000001e-06 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793825+0000", 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3300000000000004e-06 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.477 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793833+0000", 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7069999999999992e-06 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793971+0000", 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013821800000000001 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.793982+0000", 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0956e-05 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.478 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784351+0000", 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99036882999999998 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784395+0000", 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.375e-05 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.784433+0000", 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8828000000000002e-05 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.479 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105470 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.620134938999996, 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99122849999999996, 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105470 2024-05-08T03:22:03.480 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.481 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796226+0000", 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3000000000000001e-07 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796229+0000", 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0000000000000001e-06 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796237+0000", 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2389999999999996e-06 2024-05-08T03:22:03.482 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796242+0000", 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796285+0000", 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3025999999999998e-05 2024-05-08T03:22:03.483 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796302+0000", 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6368999999999999e-05 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785670+0000", 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98936853899999999 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.484 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.785718+0000", 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8232999999999997e-05 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787454+0000", 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.001736053 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105471 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.620110644999997, 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99138437000000001, 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105471 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:22:03.486 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796250+0000", 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:22:03.487 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0779999999999998e-06 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796258+0000", 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3859999999999998e-06 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.488 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796264+0000", 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6210000000000001e-06 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796353+0000", 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9786999999999999e-05 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796362+0000", 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2640000000000008e-06 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.489 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787494+0000", 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99113249299999995 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787565+0000", 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0679999999999994e-05 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787635+0000", 2024-05-08T03:22:03.490 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9676000000000002e-05 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105472 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.620037527999997, 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99148972199999996, 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.491 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105472 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796323+0000", 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.492 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9400000000000005e-07 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796327+0000", 2024-05-08T03:22:03.493 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3589999999999999e-06 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796336+0000", 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4509999999999992e-06 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.494 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796342+0000", 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9689999999999999e-06 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796388+0000", 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6279000000000002e-05 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.495 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796395+0000", 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0820000000000001e-06 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787661+0000", 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126552899999998 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787732+0000", 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1235999999999994e-05 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.496 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787813+0000", 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1223000000000005e-05 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105473 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.620009295000003, 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99163942299999996, 2024-05-08T03:22:03.497 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.498 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.498 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.498 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.498 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105473 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.499 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796352+0000", 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.32e-07 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796355+0000", 2024-05-08T03:22:03.500 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6840000000000001e-06 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796361+0000", 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650000000000002e-06 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796368+0000", 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6969999999999996e-06 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.501 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796421+0000", 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3078999999999998e-05 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2939999999999999e-06 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787844+0000", 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99141927399999996 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.502 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787923+0000", 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8948000000000004e-05 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.787991+0000", 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7749999999999993e-05 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105474 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.503 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.619978548000006, 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99188936100000002, 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105474 2024-05-08T03:22:03.504 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796382+0000", 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.505 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796383+0000", 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4200000000000001e-07 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796385+0000", 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4980000000000001e-06 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796391+0000", 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2199999999999997e-06 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.506 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796397+0000", 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1220000000000003e-06 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796447+0000", 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0661999999999997e-05 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796454+0000", 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9349999999999997e-06 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.507 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788030+0000", 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99157551600000005 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788132+0000", 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010276300000000001 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788272+0000", 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013920299999999999 2024-05-08T03:22:03.508 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105475 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.619941189000002, 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.992020915, 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.509 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105475 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:03.510 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796420+0000", 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.92e-07 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796425+0000", 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2520000000000001e-06 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.511 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796433+0000", 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.4209999999999995e-06 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796439+0000", 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2230000000000004e-06 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796476+0000", 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6295999999999999e-05 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.512 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.796482+0000", 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6560000000000003e-06 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788300+0000", 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99181797400000005 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788373+0000", 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2335000000000005e-05 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.513 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788441+0000", 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7966000000000006e-05 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.514 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105484 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 16384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618339457999994, 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99059339899999999, 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.515 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105484 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798021+0000", 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798022+0000", 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.057e-06 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798027+0000", 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8659999999999997e-06 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:22:03.517 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4345000000000001e-05 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798050+0000", 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2689999999999999e-06 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798112+0000", 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1651999999999999e-05 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.518 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798120+0000", 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3489999999999992e-06 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788464+0000", 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.990343694 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.519 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788528+0000", 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3930000000000006e-05 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788615+0000", 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7237000000000005e-05 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.520 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105485 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618285436999997, 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99072911299999999, 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.521 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105485 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798075+0000", 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.522 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798076+0000", 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8800000000000003e-07 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798079+0000", 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1860000000000001e-06 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798093+0000", 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.3473e-05 2024-05-08T03:22:03.523 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798098+0000", 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5999999999999997e-06 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798150+0000", 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2002000000000001e-05 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798159+0000", 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9590000000000001e-06 2024-05-08T03:22:03.524 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788653+0000", 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99049353200000001 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788722+0000", 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9423000000000003e-05 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788805+0000", 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2449999999999998e-05 2024-05-08T03:22:03.525 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105486 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618252730999998, 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99088611299999996, 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.526 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105486 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798108+0000", 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.527 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798109+0000", 2024-05-08T03:22:03.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2900000000000004e-07 2024-05-08T03:22:03.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.528 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798111+0000", 2024-05-08T03:22:03.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7010000000000001e-06 2024-05-08T03:22:03.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798117+0000", 2024-05-08T03:22:03.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8989999999999998e-06 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798123+0000", 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5339999999999999e-06 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798187+0000", 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3893999999999995e-05 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798191+0000", 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1799999999999998e-06 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788852+0000", 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99066087999999997 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788925+0000", 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3736e-05 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.788994+0000", 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8759999999999999e-05 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105487 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618228338999998, 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99102666699999997, 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.532 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105487 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798133+0000", 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2399999999999999e-07 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798135+0000", 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.525e-06 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798142+0000", 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1720000000000004e-06 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798147+0000", 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.023e-06 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798216+0000", 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9290999999999998e-05 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798220+0000", 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2139999999999998e-06 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789019+0000", 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99079897699999997 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789082+0000", 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2935000000000006e-05 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789159+0000", 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7106000000000003e-05 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105488 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618208917999993, 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99117645300000001, 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.537 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105488 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798152+0000", 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6e-07 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798154+0000", 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.488e-06 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798157+0000", 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798160+0000", 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.813e-06 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798240+0000", 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0465999999999998e-05 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798244+0000", 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5760000000000002e-06 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789195+0000", 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99095160199999999 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789260+0000", 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4819000000000004e-05 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789328+0000", 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8132000000000004e-05 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105489 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618197639000002, 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99135514000000002, 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.542 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105489 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798163+0000", 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0699999999999999e-07 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798165+0000", 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.6109999999999999e-06 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798168+0000", 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0769999999999999e-06 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798171+0000", 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5390000000000002e-06 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798263+0000", 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2581e-05 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798267+0000", 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7170000000000002e-06 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789375+0000", 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99110833899999995 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789450+0000", 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4658000000000002e-05 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789518+0000", 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8411000000000006e-05 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:105490 3.19 3:9ae296fd:::rbd_data.11826dcb3210.0000000000000065:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 72.618177944999999, 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.991495291, 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:03.547 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 105490 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798183+0000", 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6399999999999998e-07 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798185+0000", 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1270000000000001e-06 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798189+0000", 2024-05-08T03:22:03.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5710000000000002e-06 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798192+0000", 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7240000000000001e-06 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798280+0000", 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8134999999999996e-05 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:50.798283+0000", 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5559999999999999e-06 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789544+0000", 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99126093199999998 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789612+0000", 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7407e-05 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:03.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:51.789678+0000", 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6575000000000005e-05 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:03.552 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:03.553 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:22:03.554 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:03.551+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:03.654 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:03.652+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:03.706 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:03.707 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:03.707 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:03.707 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:03.716 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:22:03.755 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:03.753+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:03.811 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:22:03.832 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:03.833 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:03.833 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:03.833 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:03.833 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:03.842 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:22:03.856 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:03.854+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:03.956 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:03.955+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:04.001 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:04.001 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:22:04.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:22:04.001 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30353 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.898105696, 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.957134398, 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30353 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.002 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100023+0000", 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8700000000000001e-07 2024-05-08T03:22:04.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100025+0000", 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9850000000000001e-06 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100032+0000", 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4470000000000003e-06 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100037+0000", 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8799999999999999e-06 2024-05-08T03:22:04.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100327+0000", 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000290131 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.100337+0000", 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9620000000000005e-06 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057145+0000", 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95680882199999995 2024-05-08T03:22:04.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057157+0000", 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1783999999999999e-05 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30416 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89638522600001, 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95579064800000002, 2024-05-08T03:22:04.006 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30416 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101743+0000", 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101744+0000", 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9499999999999998e-07 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101753+0000", 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101759+0000", 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101764+0000", 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4649999999999996e-06 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101823+0000", 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9568000000000003e-05 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101833+0000", 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.9240000000000002e-06 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057523+0000", 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95568982199999997 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057534+0000", 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1083e-05 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.010 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30417 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89635178499999, 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95580230700000002, 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.011 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30417 2024-05-08T03:22:04.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101777+0000", 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0800000000000005e-07 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101780+0000", 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.993e-06 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101787+0000", 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0709999999999997e-06 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101793+0000", 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2270000000000001e-06 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101860+0000", 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6966999999999998e-05 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.101867+0000", 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.824e-06 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057570+0000", 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95570278099999995 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057579+0000", 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9360000000000005e-06 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.016 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30479 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89492604599999, 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95440216600000005, 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30479 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:04.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103203+0000", 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103266+0000", 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2836999999999995e-05 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103274+0000", 2024-05-08T03:22:04.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2050000000000002e-06 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103280+0000", 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7790000000000001e-06 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103458+0000", 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00017854599999999999 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103468+0000", 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0176999999999999e-05 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057596+0000", 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95412770499999999 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057605+0000", 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5010000000000004e-06 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.021 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30480 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.894833432, 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95433657999999999, 2024-05-08T03:22:04.022 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30480 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103295+0000", 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103296+0000", 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8899999999999999e-07 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103298+0000", 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.481e-06 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103304+0000", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2299999999999996e-06 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103310+0000", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5899999999999998e-06 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103495+0000", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00018531099999999999 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103503+0000", 2024-05-08T03:22:04.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6020000000000002e-06 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057622+0000", 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95411950599999995 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057632+0000", 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2709999999999998e-06 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30543 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.894322758, 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384993299999998, 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30543 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103806+0000", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8500000000000002e-07 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103812+0000", 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2060000000000004e-06 2024-05-08T03:22:04.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103818+0000", 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8510000000000004e-06 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103823+0000", 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2629999999999997e-06 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104397+0000", 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00057461399999999998 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104408+0000", 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1266e-05 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057648+0000", 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95323928899999999 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057656+0000", 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0590000000000003e-06 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.030 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30544 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89429525, 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95384690800000005, 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30544 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.031 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103833+0000", 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103834+0000", 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9700000000000002e-07 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103836+0000", 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1969999999999999e-06 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103842+0000", 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6500000000000001e-06 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.103846+0000", 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000002e-06 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104437+0000", 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00059124399999999999 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104446+0000", 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1519999999999998e-06 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057672+0000", 2024-05-08T03:22:04.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95322576000000003 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057680+0000", 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0180000000000001e-06 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30608 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.893888819, 2024-05-08T03:22:04.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95346450599999999, 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30608 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104240+0000", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104241+0000", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7599999999999996e-07 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104244+0000", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8859999999999997e-06 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.037 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104249+0000", 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6530000000000002e-06 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104253+0000", 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.703e-06 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104472+0000", 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00021935800000000001 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104481+0000", 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1419999999999999e-06 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057698+0000", 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95321703499999999 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057704+0000", 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9529999999999996e-06 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.039 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30673 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.893433441, 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95303411900000001, 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30673 2024-05-08T03:22:04.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104695+0000", 2024-05-08T03:22:04.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.65e-07 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104698+0000", 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3750000000000001e-06 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104704+0000", 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9399999999999999e-06 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104708+0000", 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4399999999999998e-06 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104758+0000", 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9443000000000001e-05 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.104767+0000", 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7040000000000006e-06 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057721+0000", 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95295359800000001 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057729+0000", 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2539999999999992e-06 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.044 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30737 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89290519399999, 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95253125800000005, 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30737 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105223+0000", 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105224+0000", 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7500000000000001e-07 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105226+0000", 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1169999999999998e-06 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105232+0000", 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1279999999999996e-06 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105236+0000", 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3640000000000001e-06 2024-05-08T03:22:04.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105283+0000", 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6443000000000002e-05 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105292+0000", 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8449999999999992e-06 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057750+0000", 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95245792799999995 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057755+0000", 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0579999999999997e-06 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30769 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:04.049 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.892432696, 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209171199999998, 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30769 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.050 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105696+0000", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8000000000000001e-07 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105698+0000", 2024-05-08T03:22:04.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0090000000000002e-06 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105704+0000", 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7239999999999997e-06 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105708+0000", 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4009999999999999e-06 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105755+0000", 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6545999999999998e-05 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105763+0000", 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1699999999999997e-06 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057779+0000", 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95201632999999997 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057788+0000", 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1519999999999992e-06 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30770 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.892354564, 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95209400399999999, 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30770 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105774+0000", 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105775+0000", 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1600000000000001e-07 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105777+0000", 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6129999999999998e-06 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105784+0000", 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8519999999999998e-06 2024-05-08T03:22:04.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105789+0000", 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9760000000000003e-06 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105831+0000", 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2525999999999999e-05 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105839+0000", 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.622e-06 2024-05-08T03:22:04.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057797+0000", 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95195791900000004 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057868+0000", 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0980000000000001e-05 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.058 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30771 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89227914200001, 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95197209699999996, 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30771 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105849+0000", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105850+0000", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.75e-07 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105853+0000", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5780000000000001e-06 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105859+0000", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2910000000000004e-06 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105865+0000", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2620000000000004e-06 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105906+0000", 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1349e-05 2024-05-08T03:22:04.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105915+0000", 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.1349999999999992e-06 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057813+0000", 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189858199999999 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057822+0000", 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3250000000000008e-06 2024-05-08T03:22:04.062 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30772 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89220197900001, 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95196555100000002, 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.063 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30772 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105927+0000", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0500000000000003e-07 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105930+0000", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9849999999999998e-06 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105937+0000", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8390000000000001e-06 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105942+0000", 2024-05-08T03:22:04.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.276e-06 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105985+0000", 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2685000000000002e-05 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.105993+0000", 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.7209999999999994e-06 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057884+0000", 2024-05-08T03:22:04.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95189174600000004 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057892+0000", 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6939999999999995e-06 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30773 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.892124456, 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192555899999998, 2024-05-08T03:22:04.067 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30773 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106004+0000", 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106005+0000", 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7400000000000003e-07 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106008+0000", 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8219999999999998e-06 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106014+0000", 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6749999999999996e-06 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106019+0000", 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2009999999999998e-06 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106708+0000", 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068883599999999996 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106716+0000", 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.0309999999999998e-06 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057920+0000", 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95120418799999995 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057930+0000", 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.2320000000000007e-06 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.071 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30774 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89209754399999, 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192274399999999, 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30774 2024-05-08T03:22:04.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106031+0000", 2024-05-08T03:22:04.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4900000000000001e-07 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106035+0000", 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4130000000000002e-06 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106041+0000", 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9920000000000004e-06 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106046+0000", 2024-05-08T03:22:04.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7169999999999999e-06 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106740+0000", 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00069410800000000003 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106747+0000", 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8739999999999997e-06 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057946+0000", 2024-05-08T03:22:04.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119957200000005 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057954+0000", 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6190000000000002e-06 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30775 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.892076334, 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95191751300000005, 2024-05-08T03:22:04.076 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30775 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106052+0000", 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106053+0000", 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106055+0000", 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.968e-06 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106060+0000", 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7869999999999998e-06 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106066+0000", 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7100000000000004e-06 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106768+0000", 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070182199999999995 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106774+0000", 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5030000000000002e-06 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057965+0000", 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119032800000003 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057970+0000", 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9810000000000003e-06 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.080 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30776 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.89205655400001, 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95192903200000001, 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30776 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106072+0000", 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106075+0000", 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106081+0000", 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7860000000000002e-06 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106093+0000", 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.253e-05 2024-05-08T03:22:04.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106796+0000", 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070303100000000001 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106803+0000", 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.5019999999999998e-06 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.057993+0000", 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95119074299999995 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058001+0000", 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.588e-06 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30821 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.891502028, 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95151960400000002, 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.085 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30821 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106627+0000", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.3299999999999998e-07 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106629+0000", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8300000000000001e-06 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106635+0000", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0569999999999998e-06 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106640+0000", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.0509999999999996e-06 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106965+0000", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000325373 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106972+0000", 2024-05-08T03:22:04.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7549999999999997e-06 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058138+0000", 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.951166387 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058146+0000", 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8180000000000004e-06 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:30822 3.6 3.c094e506 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 115.891483016, 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95153713200000001, 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 30822 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.090 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106646+0000", 2024-05-08T03:22:04.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106648+0000", 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.9819999999999998e-06 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106659+0000", 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0634e-05 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106663+0000", 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9579999999999998e-06 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.106994+0000", 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033098900000000001 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:08.107001+0000", 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3629999999999999e-06 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058173+0000", 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.95117264199999996 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:09.058183+0000", 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.1949999999999992e-06 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.094 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:04.096 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:22:04.096 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:04.094+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:04.196 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:04.194+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:04.252 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:04.252 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:04.252 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:04.252 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:04.261 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:22:04.304 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:04.303+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:04.369 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:04.369 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:04.369 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:04.369 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:04.369 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:04.379 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:22:04.398 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:04.396+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:04.499 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:04.497+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:04.527 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:04.527 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:22:04.527 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:22:04.527 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 125.528854377, 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.528 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:22:04.529 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.530 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:22:04.531 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.532 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 125.528786913, 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.533 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.534 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:22:04.535 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:22:04.536 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:22:04.537 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:04.538 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.539 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 125.468952581, 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.540 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.541 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.542 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:22:04.543 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:22:04.544 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:22:04.545 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.40244456800001, 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.546 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:22:04.548 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.549 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:22:04.550 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:22:04.551 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.375906922, 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.552 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.553 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.554 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.555 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:22:04.556 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:22:04.557 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 124.347995154, 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.558 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.559 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.269414960999995, 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.563 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.564 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:22:04.565 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264787584999993, 2024-05-08T03:22:04.566 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.567 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.568 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.569 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.570 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264766949999995, 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:22:04.571 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.572 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.573 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:22:04.574 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264744699000005, 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.575 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.576 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:22:04.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.578 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.579 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264724267999995, 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.580 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.581 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:22:04.582 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:22:04.583 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264704042000005, 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.584 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.585 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:22:04.586 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.587 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.588 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264683247999997, 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.589 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:22:04.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:22:04.592 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.264657233999998, 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.593 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:22:04.595 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.596 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.597 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.084700314000003, 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.598 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.599 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.600 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.601 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.084681209999999, 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:22:04.602 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:22:04.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:22:04.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.605 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.084119635999997, 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.606 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.607 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:22:04.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:22:04.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.078905774999996, 2024-05-08T03:22:04.610 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.611 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.612 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:22:04.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:22:04.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 96.078738434000002, 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.616 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.617 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.618 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.619 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:22:04.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.621 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 87.715984836999993, 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.622 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:22:04.623 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:04.624 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.625 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:04.626 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:04.628 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:22:04.629 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:04.627+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:04.729 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:04.727+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:04.734 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:04.734 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:04.734 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:04.735 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:04.743 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:22:04.784 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:22:04.804 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4, 7] out_osds: [0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:22:04.804 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:22:04.804 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.4810679366189673 2024-05-08T03:22:04.804 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 0.4810679366189673 2024-05-08T03:22:04.847 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:04.846+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:04.898 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:04.898 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:04.898 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:04.899 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:04.899 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:04.907 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:22:04.951 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:04.949+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:22:05.014 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.238465203999993, 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:22:05.015 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.016 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.017 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.018 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.019 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.223841999000001, 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.020 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.021 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:22:05.022 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:22:05.023 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:22:05.024 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.026 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.223032720000006, 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:22:05.027 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:22:05.028 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:22:05.029 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:22:05.030 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:22:05.031 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:22:05.032 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.222093443000006, 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.033 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.034 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.035 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.036 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.037 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.038 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.220460372999995, 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.039 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:05.040 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.041 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.042 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:22:05.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:22:05.045 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.219021877000003, 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.046 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:22:05.050 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.219001989000006, 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.051 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:22:05.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:22:05.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:22:05.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.218978070000006, 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.056 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:22:05.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.218006889999998, 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.061 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:22:05.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.065 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.217995281, 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.066 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.070 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.217985562999999, 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:22:05.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:22:05.075 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.217975100999993, 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.076 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.080 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.215398155000003, 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.081 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:22:05.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:22:05.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.215367908000005, 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.086 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:22:05.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:22:05.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.215340890999997, 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.091 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:05.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:22:05.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:22:05.097 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.097 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.097 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.097 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.215321897999999, 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:22:05.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:22:05.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.215291839000002, 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:22:05.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:22:05.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.213507375000006, 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:22:05.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:22:05.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.112 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.213495551999998, 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 74.213483694999994, 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:22:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.118 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:22:05.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:05.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:05.122 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:05.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:22:05.124 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:05.123+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:05.225 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:05.223+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:05.228 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:22:05.270 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:05.270 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:05.270 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:05.270 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:05.278 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:22:05.351 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:05.351 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:05.363 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:05.361+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:05.368 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:05.369 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:05.385 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:05.385 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:05.402 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:05.402 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:05.419 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:05.419 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:05.425 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:05.425 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:05.425 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:05.425 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:05.425 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:05.426 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:05.425+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:05.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:22:05.437 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:05.437 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:05.454 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:05.455 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:05.472 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:05.472 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:05.527 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:05.525+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:05.579 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:05.580 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:05.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:22:05.632 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:05.630+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:05.693 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:05.693 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:05.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:22:05.743 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:05.741+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:05.785 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.1 to 0.481068 (7b27) 2024-05-08T03:22:05.843 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:05.842+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:05.852 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:05.853 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:05.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:22:05.931 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:05.930+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:05.968 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:05.968 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:05.977 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:22:06.032 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:06.030+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:06.083 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.083 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.091 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:22:06.132 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:06.130+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:06.234 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:06.232+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:06.236 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.236 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:22:06.335 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:06.333+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:06.390 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.390 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.399 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:22:06.441 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:06.439+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:06.515 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.516 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.525 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:22:06.537 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:06.535+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:06.631 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.632 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.638 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:06.636+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:06.640 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:22:06.739 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:06.738+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:06.787 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.787 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.798 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:22:06.801 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T03:22:06.842 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:06.841+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:06.907 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:06.908 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:06.917 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:22:06.941 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:06.940+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:07.022 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:07.023 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:07.031 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:22:07.075 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:07.074+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:07.136 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:07.136 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:07.143 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:07.141+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:07.146 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:22:07.147 INFO:teuthology.orchestra.run.smithi177.stderr:2024-05-08T03:22:07.145+0000 7f9471a9c640 0 --2- 172.21.15.177:0/3172999909 >> v2:172.21.15.177:3300/0 conn(0x7f946c0b2940 0x7f946c13fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T03:22:07.244 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:07.242+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:07.253 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:07.253 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:07.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:22:07.346 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:07.345+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:07.408 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:07.408 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:07.417 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:22:07.458 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:07.456+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:07.483 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:22:07.547 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:07.546+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:07.564 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:07.564 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:07.574 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:22:07.625 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:07.626 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:07.644 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:07.645 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:07.648 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:07.646+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:07.661 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:07.662 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:07.679 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:07.679 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:07.697 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:07.697 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:07.715 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:07.715 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:07.723 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:07.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:07.723 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:07.723 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:07.732 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:22:07.733 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:07.733 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:07.749 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:07.747+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:07.751 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:07.751 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:07.788 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is set 2024-05-08T03:22:07.850 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:07.848+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:07.852 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:07.852 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:07.852 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:07.852 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:07.852 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:07.862 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:22:07.950 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:07.948+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:07.979 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:07.979 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:22:07.979 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22669 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch reconnect cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 128.98077777200001, 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000649363, 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22669 2024-05-08T03:22:07.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996379+0000", 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:07.981 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996381+0000", 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.8589999999999999e-06 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996388+0000", 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7259999999999996e-06 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996402+0000", 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.4491000000000001e-05 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.982 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996412+0000", 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0229e-05 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996470+0000", 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7958000000000003e-05 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op must wait for map", 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996476+0000", 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8850000000000004e-06 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999970+0000", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0034935259999999998 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.999980+0000", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0394000000000001e-05 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.994906+0000", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.99492567099999996 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995015+0000", 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000109714 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997028+0000", 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0020129100000000001 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22670 3.c 3:340dce00:::rbd_header.118227bb2f9:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 128.980710308, 2024-05-08T03:22:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.000754943, 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22670 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:22:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996446+0000", 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996447+0000", 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8599999999999998e-07 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996451+0000", 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.817e-06 2024-05-08T03:22:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996459+0000", 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.5129999999999996e-06 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996465+0000", 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.7479999999999999e-06 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996519+0000", 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4323999999999997e-05 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting_for_map not empty", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:58.996523+0000", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8229999999999997e-06 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000404+0000", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0038804489999999998 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.000413+0000", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.867e-06 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995651+0000", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.995238187 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995707+0000", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5829999999999999e-05 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.995986+0000", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00027960000000000002 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996673+0000", 2024-05-08T03:22:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00068662499999999995 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997106+0000", 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000433282 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997150+0000", 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4082000000000002e-05 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997201+0000", 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1209999999999999e-05 2024-05-08T03:22:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:22671 3.c 3:340dce00:::rbd_header.118227bb2f9:head [watch ping cookie 140222362680480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 128.92087597599999, 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.94120795899999998, 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 22671 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056281+0000", 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056282+0000", 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7899999999999997e-07 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056286+0000", 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6750000000000001e-06 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056293+0000", 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8800000000000002e-06 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056301+0000", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.8909999999999995e-06 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056361+0000", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9571000000000003e-05 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.056369+0000", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.3420000000000008e-06 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996229+0000", 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.93985990799999997 2024-05-08T03:22:07.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for rw locks", 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.996291+0000", 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1950000000000001e-05 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997235+0000", 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00094421800000000003 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997375+0000", 2024-05-08T03:22:07.996 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000139665 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:59.997489+0000", 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00011398 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23276 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.854367963, 2024-05-08T03:22:07.997 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.20136375400000001, 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23276 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.998 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122789+0000", 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.122792+0000", 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0960000000000001e-06 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149261+0000", 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.026468940999999999 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:07.999 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149285+0000", 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.408e-05 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149295+0000", 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0298e-05 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149364+0000", 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8975999999999998e-05 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.000 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149547+0000", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000182957 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.153277+0000", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0037295649999999998 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.274922+0000", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.121645399 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.001 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324061+0000", 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.049138646000000001 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324113+0000", 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1978000000000003e-05 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.324152+0000", 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9817999999999997e-05 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.002 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:23277 3.d 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.82783031699999, 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.17621137000000001, 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 23277 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.003 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149326+0000", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149328+0000", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0710000000000002e-06 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149333+0000", 2024-05-08T03:22:08.004 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.566e-06 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149338+0000", 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7330000000000003e-06 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.149341+0000", 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4970000000000001e-06 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208723+0000", 2024-05-08T03:22:08.005 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.059381597000000001 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208794+0000", 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1517e-05 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.208914+0000", 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000119521 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.284431+0000", 2024-05-08T03:22:08.006 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075517570000000006 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325446+0000", 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041014969999999998 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325482+0000", 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5188999999999998e-05 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.007 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.325538+0000", 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6138999999999999e-05 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_repop(client.4482.0:23280 3.11 e55/45 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head v 55'90, mlcod=53'88)", 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 127.799918549, 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.123014239, 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.008 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177238+0000", 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.177240+0000", 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.364e-06 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.009 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218925+0000", 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.041685119999999999 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218949+0000", 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.461e-05 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.218960+0000", 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0376e-05 2024-05-08T03:22:08.010 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219030+0000", 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.0443000000000003e-05 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.219056+0000", 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300209+0000", 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.081153457999999998 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.011 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:00.300252+0000", 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.333e-05 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:70936 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.721338356000004, 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088111520999999998, 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.012 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 70936 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255818+0000", 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255819+0000", 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.72e-07 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255821+0000", 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.024e-06 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255827+0000", 2024-05-08T03:22:08.014 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9580000000000004e-06 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.255831+0000", 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7149999999999997e-06 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292681+0000", 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036850068999999999 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.292758+0000", 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.6652999999999999e-05 2024-05-08T03:22:08.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.343930+0000", 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.051171729999999999 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71204 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716710980000002, 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075581796000000007, 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.016 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71204 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:08.017 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260446+0000", 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260449+0000", 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.34e-06 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.018 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260455+0000", 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3840000000000002e-06 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260460+0000", 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9359999999999998e-06 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335889+0000", 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075429504999999994 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.019 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.335964+0000", 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4320999999999995e-05 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336028+0000", 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3811000000000003e-05 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71205 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716690345000004, 2024-05-08T03:22:08.020 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075745988, 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71205 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.021 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260466+0000", 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260467+0000", 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9900000000000001e-07 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260469+0000", 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.221e-06 2024-05-08T03:22:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260475+0000", 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2920000000000001e-06 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260480+0000", 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8949999999999997e-06 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336054+0000", 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075573990999999993 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.023 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336147+0000", 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.3183000000000006e-05 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336212+0000", 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4906999999999994e-05 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71206 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716668093999999, 2024-05-08T03:22:08.024 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075888238999999996, 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71206 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260489+0000", 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260491+0000", 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2929999999999999e-06 2024-05-08T03:22:08.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260498+0000", 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6000000000000003e-06 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260503+0000", 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6870000000000002e-06 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336240+0000", 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075737402999999995 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.027 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336313+0000", 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3088000000000002e-05 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336377+0000", 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3751999999999995e-05 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.028 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71207 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 806912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716647663000003, 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076032833999999994, 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71207 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.029 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260509+0000", 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0999999999999999e-07 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.030 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260512+0000", 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2639999999999998e-06 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260518+0000", 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2469999999999997e-06 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260523+0000", 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9470000000000002e-06 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.031 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336403+0000", 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.075880366000000005 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336478+0000", 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4441000000000001e-05 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336542+0000", 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4158999999999993e-05 2024-05-08T03:22:08.032 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71208 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716627437, 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076177247000000003, 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.033 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71208 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260529+0000", 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.034 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260530+0000", 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.6899999999999998e-07 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260532+0000", 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3010000000000001e-06 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260538+0000", 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.0610000000000001e-06 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.035 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260543+0000", 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7829999999999998e-06 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336569+0000", 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076026363 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336643+0000", 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.4128000000000006e-05 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.036 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336707+0000", 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3242000000000002e-05 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71209 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716606643000006, 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076320581999999998, 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.037 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71209 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260550+0000", 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.89e-07 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260557+0000", 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.1870000000000002e-06 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260564+0000", 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1899999999999998e-06 2024-05-08T03:22:08.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260569+0000", 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2469999999999999e-06 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336734+0000", 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076165369999999996 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336806+0000", 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1824999999999998e-05 2024-05-08T03:22:08.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.336871+0000", 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4374000000000004e-05 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:71210 3.18 3:1e2721af:::rbd_data.11828f9dff47.0000000000000057:22 [read 995328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.716580629000006, 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.077016756000000006, 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.041 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 71210 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.042 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260576+0000", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260577+0000", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260579+0000", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4569999999999999e-06 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260586+0000", 2024-05-08T03:22:08.043 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9720000000000003e-06 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.260591+0000", 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1749999999999996e-06 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337433+0000", 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.076842167000000003 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337521+0000", 2024-05-08T03:22:08.044 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7650000000000003e-05 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.337593+0000", 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1890000000000005e-05 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73452 3.8 3:142d6589:::rbd_data.11828c8e13f1.0000000000000019:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.536623708999997, 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.087648951000000003, 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.045 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73452 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.046 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440533+0000", 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440534+0000", 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3900000000000005e-07 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440536+0000", 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.137e-06 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.047 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440542+0000", 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.8579999999999996e-06 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440546+0000", 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4259999999999996e-06 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512160+0000", 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071614428999999993 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.048 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512293+0000", 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000132714 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.528182+0000", 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.015888848000000001 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.049 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73453 3.8 3:105ff9c2:::rbd_data.11828c8e13f1.000000000000001a:head [stat,delete] snapc 23=[23] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.536604604999994, 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0896763, 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73453 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.050 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440552+0000", 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440553+0000", 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3099999999999998e-07 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440555+0000", 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.091e-06 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440561+0000", 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3620000000000002e-06 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.440566+0000", 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9799999999999998e-06 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.052 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513103+0000", 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.072536524000000005 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513284+0000", 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.000180929 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530228+0000", 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016944883000000001 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.053 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73457 3.18 3:18f6bf4a:::rbd_header.11828c8e13f1:head [watch reconnect cookie 140222294580656 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.536043031000005, 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.088931575999999998, 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73457 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.054 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441114+0000", 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.75e-07 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.055 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441116+0000", 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0389999999999999e-06 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441122+0000", 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.9780000000000002e-06 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.441127+0000", 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4599999999999996e-06 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.056 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512710+0000", 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.071583784999999997 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.512773+0000", 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.2650999999999994e-05 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530045+0000", 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017272188000000001 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.057 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73461 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.530829170000004, 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083905010000000002, 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73461 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.058 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446328+0000", 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446329+0000", 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.175e-06 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.059 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446458+0000", 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00012951299999999999 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446469+0000", 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0474e-05 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446477+0000", 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.7590000000000003e-06 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.060 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513213+0000", 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.066735645999999996 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513281+0000", 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.8214999999999997e-05 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.513990+0000", 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00070876200000000004 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.061 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.522294+0000", 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0083041950000000003 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530151+0000", 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0078570759999999993 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530198+0000", 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6940000000000001e-05 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.062 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530233+0000", 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4255000000000003e-05 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:73462 3.18 3:1f72e191:::rbd_data.11828c8e13f1.0000000000000011:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 99.530661828999996, 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.083857685000000001, 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.063 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 73462 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446495+0000", 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446496+0000", 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3900000000000004e-07 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446498+0000", 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446506+0000", 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.3459999999999996e-06 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.446511+0000", 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4199999999999998e-06 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.516948+0000", 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.070437110999999997 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517015+0000", 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.6505000000000001e-05 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 1", 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.517175+0000", 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00016003 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.524049+0000", 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0068746720000000001 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.067 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530279+0000", 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0062294550000000001 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530320+0000", 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1084999999999997e-05 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:28.530353+0000", 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.2521999999999998e-05 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.068 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:100997 3.7 3:e17728eb:::image_client.0-clone36.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 91.167908232000002, 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98106100100000004, 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "reached pg", 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 100997 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.069 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809248+0000", 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:08.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809249+0000", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809252+0000", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.864e-06 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809258+0000", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.3659999999999997e-06 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809262+0000", 2024-05-08T03:22:08.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8160000000000004e-06 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809313+0000", 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1122000000000003e-05 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:36.809321+0000", 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.435e-06 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790206+0000", 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.98088530100000004 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:08.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:20:37.790309+0000", 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00010365300000000001 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:08.073 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:08.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:22:08.075 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:08.073+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:08.175 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:08.173+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:08.196 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:08.196 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:08.196 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:08.196 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:08.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:22:08.276 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:08.276+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:08.360 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:08.360 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:08.360 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:08.360 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:08.360 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:08.370 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:22:08.377 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:08.376+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:08.478 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:08.476+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:08.497 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105377 3.d 3:b39ff687:::rbd_header.1182b2d87865:head [watch reconnect cookie 140222362193312 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.721797901000002, 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.014913857, 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.498 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105377 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.499 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773072+0000", 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773073+0000", 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8400000000000003e-07 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773077+0000", 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0709999999999996e-06 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.500 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773086+0000", 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7950000000000002e-06 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773093+0000", 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8689999999999998e-06 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773141+0000", 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7519999999999999e-05 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.501 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op must wait for map", 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.773145+0000", 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.442e-06 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779567+0000", 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0064212109999999996 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.779577+0000", 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0002e-05 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.502 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784449+0000", 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0048726539999999 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.784588+0000", 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013850900000000001 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787986+0000", 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0033979000000000001 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.503 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105379 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.707174695999996, 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0012236670000001, 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.504 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105379 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:22:08.505 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787695+0000", 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787696+0000", 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.61e-07 2024-05-08T03:22:08.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787699+0000", 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8199999999999998e-06 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787708+0000", 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.7919999999999998e-06 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.787713+0000", 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0100000000000003e-06 2024-05-08T03:22:08.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791788+0000", 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0040752519999999997 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791793+0000", 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.809e-06 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785442+0000", 2024-05-08T03:22:08.508 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99364900899999997 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.785572+0000", 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00012982099999999999 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786219+0000", 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000647027 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.509 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787869+0000", 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0016496410000000001 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788833+0000", 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00096409600000000001 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788880+0000", 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6631999999999997e-05 2024-05-08T03:22:08.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788919+0000", 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9097000000000002e-05 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105380 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.706365417000001, 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00140425, 2024-05-08T03:22:08.511 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105380 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:22:08.512 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788504+0000", 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788505+0000", 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9999999999999998e-07 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788508+0000", 2024-05-08T03:22:08.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9000000000000002e-06 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788513+0000", 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5310000000000001e-06 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.788517+0000", 2024-05-08T03:22:08.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9500000000000003e-06 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791983+0000", 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003466275 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791992+0000", 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4379999999999995e-06 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786801+0000", 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.994808888 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.786893+0000", 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.2412999999999998e-05 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.787249+0000", 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00035578299999999998 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788955+0000", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0017061999999999999 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789832+0000", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00087650800000000002 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789865+0000", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2923999999999999e-05 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789908+0000", 2024-05-08T03:22:08.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3940000000000003e-05 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105384 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.70542614, 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0009047289999999, 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.518 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105384 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789443+0000", 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789444+0000", 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.36e-07 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789445+0000", 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.494e-06 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789448+0000", 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1439999999999999e-06 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.520 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.789451+0000", 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792439+0000", 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.002988479 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792447+0000", 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.875e-06 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788040+0000", 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99559306999999997 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788110+0000", 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9843000000000002e-05 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:08.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788326+0000", 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00021582 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790213+0000", 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0018867199999999999 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790248+0000", 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5178000000000001e-05 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:08.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790314+0000", 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6011000000000002e-05 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790348+0000", 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4298000000000003e-05 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105397 3.d 3:b50fbece:::rbd_object_map.1182b2d87865.000000000000002a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:08.524 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.703793070000003, 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99992431699999995, 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105397 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791077+0000", 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1099999999999996e-07 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791079+0000", 2024-05-08T03:22:08.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2089999999999999e-06 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791085+0000", 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5809999999999996e-06 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.791088+0000", 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5260000000000002e-06 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793281+0000", 2024-05-08T03:22:08.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021921359999999999 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793298+0000", 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7700999999999999e-05 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.788999+0000", 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99570088800000001 2024-05-08T03:22:08.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789081+0000", 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1978999999999996e-05 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.789424+0000", 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00034315099999999998 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790941+0000", 2024-05-08T03:22:08.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0015166470000000001 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790949+0000", 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4169999999999992e-06 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.790979+0000", 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9621e-05 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791001+0000", 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1950000000000002e-05 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105419 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.702354573999997, 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871854900000001, 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.531 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105419 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792515+0000", 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1899999999999998e-07 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792518+0000", 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2680000000000001e-06 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792523+0000", 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.823e-06 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792529+0000", 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6130000000000003e-06 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795338+0000", 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028093480000000001 2024-05-08T03:22:08.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795342+0000", 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6500000000000002e-06 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791042+0000", 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99569957899999995 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791145+0000", 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000103526 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791234+0000", 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8423000000000004e-05 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105420 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.702334686, 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99883368500000003, 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.536 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105420 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:08.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792535+0000", 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3099999999999999e-07 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792537+0000", 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8729999999999999e-06 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792543+0000", 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5500000000000002e-06 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792547+0000", 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2509999999999996e-06 2024-05-08T03:22:08.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795372+0000", 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028249210000000002 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795380+0000", 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8250000000000005e-06 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791265+0000", 2024-05-08T03:22:08.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99588569000000005 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791319+0000", 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3715999999999999e-05 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791369+0000", 2024-05-08T03:22:08.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9527999999999997e-05 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105421 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.702310767, 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99895068200000003, 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.542 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105421 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792559+0000", 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7399999999999998e-07 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792562+0000", 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3599999999999999e-06 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792568+0000", 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3119999999999999e-06 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.792574+0000", 2024-05-08T03:22:08.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7039999999999999e-06 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795420+0000", 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0028467729999999999 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795426+0000", 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6999999999999996e-06 2024-05-08T03:22:08.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791406+0000", 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99597933500000002 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791458+0000", 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.24e-05 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.791510+0000", 2024-05-08T03:22:08.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1623999999999999e-05 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105433 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.701339587000007, 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99882497800000003, 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.548 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105433 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793530+0000", 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2999999999999999e-07 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793531+0000", 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.068e-06 2024-05-08T03:22:08.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793535+0000", 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7220000000000001e-06 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793538+0000", 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2459999999999999e-06 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795589+0000", 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020504049999999999 2024-05-08T03:22:08.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795596+0000", 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7579999999999992e-06 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792171+0000", 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99657457699999996 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792264+0000", 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.3400999999999996e-05 2024-05-08T03:22:08.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792355+0000", 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0570999999999997e-05 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105434 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.701327977999995, 2024-05-08T03:22:08.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99904233600000003, 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105434 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:08.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793542+0000", 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2499999999999999e-07 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793543+0000", 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.062e-06 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793546+0000", 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.948e-06 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793548+0000", 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5450000000000002e-06 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795621+0000", 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020722850000000001 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795625+0000", 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6979999999999997e-06 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792406+0000", 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99678018999999995 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792495+0000", 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9784000000000006e-05 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792584+0000", 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8598999999999997e-05 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105435 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 884736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.558 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.701318259999994, 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925947599999998, 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105435 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793551+0000", 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793552+0000", 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793553+0000", 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.049e-06 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793556+0000", 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4759999999999998e-06 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793559+0000", 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6170000000000001e-06 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795655+0000", 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0020963819999999999 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795662+0000", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1829999999999997e-06 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792632+0000", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99697012500000004 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792730+0000", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7405999999999995e-05 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792811+0000", 2024-05-08T03:22:08.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.1030000000000002e-05 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105436 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.701307798000002, 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99939996399999997, 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.564 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105436 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:08.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793562+0000", 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1e-07 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793563+0000", 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.065e-06 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793567+0000", 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8890000000000004e-06 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.793570+0000", 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6359999999999998e-06 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795682+0000", 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00211254 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.795686+0000", 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2379999999999999e-06 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792852+0000", 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99716563899999999 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792909+0000", 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6703999999999998e-05 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.792962+0000", 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3043e-05 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105464 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:08.569 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.698730851999997, 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99877308799999998, 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105464 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.570 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796139+0000", 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4499999999999997e-07 2024-05-08T03:22:08.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796142+0000", 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3420000000000002e-06 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796148+0000", 2024-05-08T03:22:08.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9140000000000002e-06 2024-05-08T03:22:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796157+0000", 2024-05-08T03:22:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0399999999999993e-06 2024-05-08T03:22:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796203+0000", 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6316000000000002e-05 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796210+0000", 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1629999999999999e-06 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794744+0000", 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853392500000004 2024-05-08T03:22:08.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794828+0000", 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4421999999999999e-05 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794912+0000", 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105465 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.698700604999999, 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99893463599999999, 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105465 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796169+0000", 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7099999999999997e-07 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796171+0000", 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1059999999999998e-06 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796178+0000", 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4130000000000003e-06 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796183+0000", 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.2599999999999996e-06 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796253+0000", 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0087999999999997e-05 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796259+0000", 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5110000000000003e-06 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.794950+0000", 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99869101900000001 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795039+0000", 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8935000000000001e-05 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795104+0000", 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4932999999999996e-05 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105466 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.585 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.698673588000005, 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99911071500000004, 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105466 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796196+0000", 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.96e-07 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796198+0000", 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9149999999999999e-06 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796204+0000", 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0109999999999997e-06 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796208+0000", 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7680000000000002e-06 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796291+0000", 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3158000000000006e-05 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796299+0000", 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8720000000000002e-06 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795144+0000", 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99884453900000003 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795230+0000", 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6007999999999994e-05 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795307+0000", 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7047999999999998e-05 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.590 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105467 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 753664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.698654594999994, 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99925422799999997, 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105467 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796215+0000", 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4200000000000002e-07 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796217+0000", 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9599999999999999e-06 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796223+0000", 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6869999999999999e-06 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796227+0000", 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1989999999999999e-06 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796324+0000", 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6618000000000002e-05 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796328+0000", 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9519999999999996e-06 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795342+0000", 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99901413100000003 2024-05-08T03:22:08.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795422+0000", 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0482000000000006e-05 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795469+0000", 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6857000000000002e-05 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.595 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105468 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.698624535999997, 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.999423642, 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105468 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796245+0000", 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796246+0000", 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1399999999999997e-07 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796247+0000", 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.846e-06 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796251+0000", 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.569e-06 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796254+0000", 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.734e-06 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796351+0000", 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.7242000000000002e-05 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.796358+0000", 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2529999999999998e-06 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795508+0000", 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99915012400000003 2024-05-08T03:22:08.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795593+0000", 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4707000000000001e-05 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.795669+0000", 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.5753000000000005e-05 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.600 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105481 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.696840072000001, 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998660506, 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105481 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798030+0000", 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798031+0000", 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3859999999999999e-06 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798034+0000", 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2930000000000001e-06 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798037+0000", 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6199999999999999e-06 2024-05-08T03:22:08.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798139+0000", 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000101958 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798144+0000", 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4620000000000004e-06 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796581+0000", 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99843625999999996 2024-05-08T03:22:08.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796632+0000", 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0899000000000002e-05 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796690+0000", 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8409999999999998e-05 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105482 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.696828249000006, 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99885416900000001, 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105482 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798041+0000", 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798042+0000", 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798043+0000", 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1999999999999999e-06 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798046+0000", 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2969999999999999e-06 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798049+0000", 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5579999999999999e-06 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798181+0000", 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00013216200000000001 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798188+0000", 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.542e-06 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796727+0000", 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99853872600000004 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796812+0000", 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5413000000000004e-05 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796896+0000", 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3065999999999994e-05 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.610 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:105483 3.d 3:b6dd479d:::rbd_data.11826dcb3210.0000000000000064:29 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 77.696816392000002, 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.998980431, 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 105483 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:08.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798053+0000", 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:08.612 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798055+0000", 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6029999999999999e-06 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798059+0000", 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3610000000000003e-06 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798064+0000", 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4159999999999997e-06 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798209+0000", 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000144788 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:50.798213+0000", 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.929e-06 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796926+0000", 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99871382099999995 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:08.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.796986+0000", 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9509999999999998e-05 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:51.797034+0000", 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7797999999999999e-05 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:08.615 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:08.617 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:22:08.617 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:08.615+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:08.717 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:08.715+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:08.770 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:08.771 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:08.771 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:08.771 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:08.779 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:22:08.819 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:08.817+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:08.919 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:08.919+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:08.926 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:08.927 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:08.927 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:08.927 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:08.927 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:08.935 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:22:09.020 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:09.019+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:09.042 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 152.83202430599999, 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.043 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.044 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.045 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:22:09.046 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:22:09.047 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.048 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.912293995, 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.049 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.050 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.051 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.052 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.91226227800001, 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.053 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:22:09.054 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:22:09.055 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.056 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.91223717400001, 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.057 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:22:09.058 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:22:09.059 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.060 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.912212369, 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:22:09.061 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:22:09.062 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.063 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:22:09.064 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.912183825, 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.065 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.066 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:22:09.067 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:22:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.912161303, 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:22:09.069 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.070 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:22:09.071 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:22:09.072 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.073 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.91213096300001, 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.074 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:22:09.075 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:22:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:22:09.077 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 151.91210978199999, 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.078 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.079 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.080 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.081 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 138.09946784600001, 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.082 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.083 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.084 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:22:09.085 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.086 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.087 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:09.088 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.910793125, 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:09.089 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.090 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.091 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.092 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.093 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.888589782, 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.094 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.095 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.096 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.097 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.098 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.88689366400001, 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.099 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.100 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.101 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.867188876, 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.103 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.104 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.105 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.106 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.107 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.86716491600001, 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.108 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.109 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.110 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.111 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.112 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.113 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.82657337099999, 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:22:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:22:09.115 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.116 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.117 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 128.81379947400001, 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.118 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:22:09.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:22:09.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:22:09.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 109.55306054, 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:22:09.122 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.123 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.124 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.125 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:22:09.126 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.194756140999999, 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:22:09.127 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:22:09.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:22:09.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.130 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 54.193137403999998, 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.131 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.132 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.133 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.134 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.135 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.135 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:09.136 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:22:09.137 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:09.134+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:09.237 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:09.236+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:09.242 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:09.242 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:09.242 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:09.242 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:09.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:22:09.338 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:09.336+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:09.356 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:09.357 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:09.357 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:09.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:22:09.439 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:09.438+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:09.513 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:09.513 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:09.513 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:09.513 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:22:09.513 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 157.63683190899999, 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:22:09.516 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.517 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.518 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.519 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29481113700001, 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.520 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.521 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:22:09.522 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:22:09.523 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.294801616, 2024-05-08T03:22:09.524 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.525 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:22:09.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:22:09.527 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.528 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29479227499999, 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.529 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.530 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.531 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.532 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29478305999999, 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:22:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:22:09.534 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:22:09.535 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.536 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.294773698, 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.537 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.538 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.539 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:22:09.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29476426599999, 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:22:09.541 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.542 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.544 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:09.545 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29475485, 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:22:09.547 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:22:09.548 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.549 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29474583199999, 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.550 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.551 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.552 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.553 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29473678900001, 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.554 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.555 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.556 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.557 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29472449599999, 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.558 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.559 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:22:09.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:22:09.561 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.29471522899999, 2024-05-08T03:22:09.562 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.563 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:22:09.564 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:22:09.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.566 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.294701371, 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.567 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.568 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:22:09.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:22:09.570 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.294692078, 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.571 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:22:09.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:22:09.574 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:144875 3.14 3.e36d51d4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:59.785375+0000", 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.7255891539999997, 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00537221, 2024-05-08T03:22:09.575 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "reached pg", 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 144875 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785375+0000", 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.576 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785375+0000", 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785377+0000", 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2110000000000001e-06 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785386+0000", 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785414+0000", 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8807e-05 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785427+0000", 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2452999999999999e-05 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.788579+0000", 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003151839 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.788590+0000", 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1430999999999999e-05 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790727+0000", 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.002136973 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790747+0000", 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0273000000000001e-05 2024-05-08T03:22:09.579 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:144876 3.14 3.e36d51d4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:59.791087+0000", 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.7198771790000009, 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99971595199999996, 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "reached pg", 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.580 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 144876 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791087+0000", 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791087+0000", 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791089+0000", 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5889999999999999e-06 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791094+0000", 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0649999999999998e-06 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791104+0000", 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0071000000000001e-05 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791111+0000", 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4909999999999999e-06 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791312+0000", 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020073000000000001 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791320+0000", 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3110000000000006e-06 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790794+0000", 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99947371699999998 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790803+0000", 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9779999999999994e-06 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.584 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:145167 3.15 3:adc0bd6f:::rbd_header.1182ca0d3624:head [watch reconnect cookie 140222361533216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:03.811167+0000", 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.6997973289999999, 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99499526199999999, 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 145167 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811167+0000", 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811167+0000", 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811169+0000", 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5090000000000001e-06 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811175+0000", 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1480000000000003e-06 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811231+0000", 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6064999999999999e-05 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811261+0000", 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9816999999999999e-05 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811798+0000", 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053767099999999998 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811807+0000", 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3990000000000004e-06 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.799321+0000", 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.98751383599999998 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.799851+0000", 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053066799999999998 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806162+0000", 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063111490000000003 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.589 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:145168 3.15 3:adc0bd6f:::rbd_header.1182ca0d3624:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:03.811587+0000", 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.6993773330000002, 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99473379799999995, 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 145168 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811587+0000", 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811587+0000", 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811588+0000", 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6199999999999997e-07 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811592+0000", 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5279999999999997e-06 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811601+0000", 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4810000000000006e-06 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811608+0000", 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.063e-06 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811831+0000", 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000222644 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811836+0000", 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8629999999999996e-06 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.800614+0000", 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.98877804599999997 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.800704+0000", 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9773999999999998e-05 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.800944+0000", 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000239418 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.805347+0000", 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0044031679999999998 2024-05-08T03:22:09.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806203+0000", 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00085625000000000002 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806267+0000", 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4183000000000005e-05 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.595 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806321+0000", 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3618000000000001e-05 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:145169 3.15 3:adc0bd6f:::rbd_header.1182ca0d3624:head [watch ping cookie 140222361533216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:04.078332+0000", 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.4326321029999995, 2024-05-08T03:22:09.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.72815680000000005, 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 145169 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078332+0000", 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078332+0000", 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078333+0000", 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1710000000000001e-06 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078338+0000", 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.357e-06 2024-05-08T03:22:09.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078351+0000", 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2685e-05 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078358+0000", 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1339999999999998e-06 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078412+0000", 2024-05-08T03:22:09.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3999999999999998e-05 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078421+0000", 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0149999999999992e-06 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.801238+0000", 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.72281739099999998 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for rw locks", 2024-05-08T03:22:09.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.801311+0000", 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2501000000000003e-05 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806349+0000", 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050382530000000003 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806409+0000", 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0676e-05 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806489+0000", 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9616999999999993e-05 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4175.0:12 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:04.210739+0000", 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 5.3002253899999996, 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.58313978099999997, 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "reached pg", 2024-05-08T03:22:09.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4175", 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3444697585", 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210739+0000", 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210739+0000", 2024-05-08T03:22:09.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210740+0000", 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9800000000000004e-07 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210743+0000", 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3380000000000001e-06 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210749+0000", 2024-05-08T03:22:09.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5200000000000003e-06 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210753+0000", 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7179999999999998e-06 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210790+0000", 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6514000000000003e-05 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:09.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210795+0000", 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8000000000000004e-06 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.793613+0000", 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.58281794499999995 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.793879+0000", 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000265448 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:09.608 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:09.608 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:09.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:22:09.610 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:09.607+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:09.710 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:09.709+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:09.754 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:09.754 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:09.755 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:09.755 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:09.763 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:22:09.764 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:22:09.810 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:09.808+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:09.811 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T03:22:09.884 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:09.884 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:09.884 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:09.884 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:09.884 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:09.894 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:22:09.912 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:09.912 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:09.931 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:09.932 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:09.935 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:09.933+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:09.954 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:09.954 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:09.971 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:09.971 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:09.989 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:09.990 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:10.007 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:10.007 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:10.012 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:10.011+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:10.025 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:10.025 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:10.042 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.042 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.043 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:10.043 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:10.052 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:22:10.113 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:10.111+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:10.157 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.157 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.166 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:22:10.213 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:10.212+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:10.313 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.313 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.314 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:10.312+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:10.322 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:22:10.414 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:10.412+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:10.468 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.468 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:22:10.521 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:10.520+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:10.583 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.583 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.592 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:22:10.616 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:10.615+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:10.717 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:10.716+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:10.738 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.739 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.748 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:22:10.802 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 6, 4, 7] out_osds: [0, 5, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-05-08T03:22:10.802 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T03:22:10.803 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-05-08T03:22:10.803 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-05-08T03:22:10.815 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is set 2024-05-08T03:22:10.818 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:10.816+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:10.868 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:10.868 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:10.878 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:22:10.919 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:10.918+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:11.020 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:11.018+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:11.030 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.030 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.039 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:22:11.121 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:11.119+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:11.145 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.146 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:22:11.222 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:11.220+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:11.301 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.301 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:22:11.354 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:11.352+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:11.415 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.415 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.423 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:11.421+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:11.424 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:22:11.524 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:11.522+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:11.568 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.569 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.577 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:22:11.661 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:11.661+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:11.723 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.723 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.725 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:11.725+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:11.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:22:11.816 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:22:11.815+0000 7fc7deb41640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-05-08T03:22:11.820 INFO:teuthology.orchestra.run.smithi177.stderr:osd set-full-ratio 0.001 2024-05-08T03:22:11.826 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:11.825+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:11.863 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.864 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:11.874 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:22:11.927 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:11.927+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:11.995 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:11.995 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:12.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:22:12.049 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:12.047+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:12.057 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:22:12.110 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:12.111 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:12.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:22:12.129 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:12.129+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:12.180 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:12.180 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:12.197 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:12.198 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:12.214 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:12.215 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:12.230 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:12.228+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:12.231 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:12.231 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:12.249 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:12.249 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:12.265 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:12.265 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:12.266 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:12.266 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:12.274 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:22:12.283 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:12.283 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:12.301 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:12.301 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:12.358 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:12.356+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:12.421 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:12.421 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:12.421 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:12.421 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:12.421 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:12.429 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:22:12.470 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:12.468+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:12.532 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:12.530+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12445 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12859034~16501094] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:22:12.576 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 156.36534517499999, 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.034071371000000003, 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12445 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.577 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208419+0000", 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.208420+0000", 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.085e-06 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.578 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219436+0000", 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01101594 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219475+0000", 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8359000000000003e-05 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219494+0000", 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8955999999999999e-05 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.579 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219610+0000", 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000116013 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.219831+0000", 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000221151 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 0", 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.220567+0000", 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00073641899999999998 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.580 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.238553+0000", 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01798543 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242371+0000", 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0038186729999999999 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242434+0000", 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2937999999999999e-05 2024-05-08T03:22:12.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:36.242491+0000", 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6406999999999997e-05 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12909 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 524288~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.44561486399999, 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016704453000000001, 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.582 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12909 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.583 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128150+0000", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128159+0000", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4290000000000001e-06 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128166+0000", 2024-05-08T03:22:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9049999999999997e-06 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128173+0000", 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999997e-06 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144481+0000", 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016308494 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144535+0000", 2024-05-08T03:22:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3189000000000002e-05 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144854+0000", 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031946399999999999 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12910 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2621440~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.44558314700001, 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017121398999999999, 2024-05-08T03:22:12.586 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12910 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128181+0000", 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128182+0000", 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6899999999999998e-07 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128184+0000", 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2069999999999998e-06 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128192+0000", 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0430000000000008e-06 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128198+0000", 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.2740000000000004e-06 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144883+0000", 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.016684732000000001 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.144930+0000", 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6406999999999998e-05 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145303+0000", 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00037326699999999999 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12925 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.44555804300001, 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017671719999999998, 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12925 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128206+0000", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128207+0000", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7300000000000002e-07 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128209+0000", 2024-05-08T03:22:12.592 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0219999999999999e-06 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128217+0000", 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9850000000000006e-06 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128223+0000", 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4989999999999999e-06 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145337+0000", 2024-05-08T03:22:12.593 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017113690000000001 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145382+0000", 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4666000000000002e-05 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145878+0000", 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00049648499999999996 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.594 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12926 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2752512~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.445533238, 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018223362999999999, 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12926 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.595 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128231+0000", 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128232+0000", 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7999999999999996e-07 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.596 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128234+0000", 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1069999999999999e-06 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128242+0000", 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0360000000000006e-06 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128247+0000", 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3009999999999997e-06 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.597 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145902+0000", 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.017655218 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.145986+0000", 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3521000000000002e-05 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146455+0000", 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00046870000000000001 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.598 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12941 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.44550469399999, 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018628986, 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12941 2024-05-08T03:22:12.599 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128260+0000", 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4400000000000001e-07 2024-05-08T03:22:12.600 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128263+0000", 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6189999999999998e-06 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128271+0000", 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7800000000000001e-06 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128276+0000", 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.011e-06 2024-05-08T03:22:12.601 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146480+0000", 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018204778000000001 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146576+0000", 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5487000000000002e-05 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146889+0000", 2024-05-08T03:22:12.602 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00031286699999999998 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12942 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 2883584~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.445482172, 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019510939000000001, 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.603 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12942 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128282+0000", 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.604 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128283+0000", 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.1600000000000002e-07 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128285+0000", 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.945e-06 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128292+0000", 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0049999999999998e-06 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.605 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128305+0000", 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3624e-05 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.146925+0000", 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.01861987 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147009+0000", 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3786e-05 2024-05-08T03:22:12.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147793+0000", 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00078429300000000003 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12957 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 917504~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.445451832, 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021736366, 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.607 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12957 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.608 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128313+0000", 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7199999999999999e-07 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128315+0000", 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1490000000000001e-06 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.609 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128323+0000", 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.25e-06 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128328+0000", 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0699999999999997e-06 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147828+0000", 2024-05-08T03:22:12.610 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019500000999999999 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.147885+0000", 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7608999999999997e-05 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150049+0000", 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0021638149999999999 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.611 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:12958 3.10 3:08263a36:::rbd_data.1182ca0d3624.0000000000000004:a [sparse-read 3014656~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 155.44543065100001, 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.022325864000000001, 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12958 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.612 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128334+0000", 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4999999999999998e-07 2024-05-08T03:22:12.613 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128337+0000", 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.858e-06 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128347+0000", 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.5670000000000001e-06 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.128352+0000", 2024-05-08T03:22:12.614 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.1460000000000004e-06 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150091+0000", 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021739472999999999 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150178+0000", 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6812999999999997e-05 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.615 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:37.150660+0000", 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00048155700000000002 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:20953 3.11 3:8abf6ad1:::rbd_data.1182cef43756.0000000000000005:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 141.632788715, 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019429697999999999, 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.616 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 20953 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940976+0000", 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.617 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940977+0000", 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2300000000000001e-06 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.940982+0000", 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5630000000000002e-06 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941004+0000", 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2121e-05 2024-05-08T03:22:12.618 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941010+0000", 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1750000000000002e-06 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.941073+0000", 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.3306999999999995e-05 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for degraded object", 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.944635+0000", 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003561584 2024-05-08T03:22:12.619 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956274+0000", 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.011639198 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956359+0000", 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.5079000000000003e-05 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.956506+0000", 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000147365 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.620 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.957740+0000", 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0012337820000000001 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960263+0000", 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0025225830000000001 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960319+0000", 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6051999999999998e-05 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.621 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:50.960406+0000", 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6658999999999998e-05 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23280 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.44411399399999, 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.170840513, 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.622 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23280 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.623 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.129651+0000", 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1600000000000001e-07 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173185+0000", 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.043534122000000001 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.624 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173211+0000", 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5537999999999998e-05 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173223+0000", 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.201e-05 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173303+0000", 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.0443000000000002e-05 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.625 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00017270499999999999 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.177055+0000", 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0035787969999999999 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.279009+0000", 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.10195452100000001 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.626 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300402+0000", 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021392121 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300451+0000", 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9379000000000003e-05 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.300491+0000", 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0160999999999997e-05 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.627 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23274 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'156, mlcod=55'154)", 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.42191065099999, 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.140893977, 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:22:12.628 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151854+0000", 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.151855+0000", 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.297e-06 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213789+0000", 2024-05-08T03:22:12.629 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.061933314000000003 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213806+0000", 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7900000000000001e-05 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213815+0000", 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9959999999999999e-06 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.630 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213881+0000", 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5409999999999999e-05 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213906+0000", 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5511e-05 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292711+0000", 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.078804686999999998 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.631 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.292748+0000", 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6862e-05 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23276 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'69, mlcod=39'67)", 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.42021453300001, 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.17023685599999999, 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.632 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153550+0000", 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.633 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.153551+0000", 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7300000000000005e-07 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226584+0000", 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.073032851999999995 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226602+0000", 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.8655999999999999e-05 2024-05-08T03:22:12.634 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226614+0000", 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2077e-05 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226679+0000", 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4449000000000006e-05 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226717+0000", 2024-05-08T03:22:12.635 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8232000000000003e-05 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323740+0000", 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097022824999999993 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.323787+0000", 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6992e-05 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.636 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23281 3.11 3:89f00afe:::rbd_data.118227bb2f9.0000000000000004:head [truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.40050974499999, 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.13193522599999999, 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23281 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.637 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173255+0000", 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173256+0000", 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1230000000000001e-06 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173259+0000", 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4809999999999998e-06 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173266+0000", 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7220000000000002e-06 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173270+0000", 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6669999999999998e-06 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.639 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223869+0000", 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.050598820000000003 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.223947+0000", 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.8580999999999999e-05 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 3", 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.224098+0000", 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000151135 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.640 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.301165+0000", 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.077066491000000001 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305131+0000", 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0039666079999999999 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.641 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305165+0000", 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3744000000000001e-05 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.305190+0000", 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4854e-05 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:23282 3.a 3:56896b78:::rbd_data.118227bb2f9.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 16777216] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-05-08T03:22:12.642 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.400485785, 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.040744692999999998, 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 23282 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.643 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173279+0000", 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6400000000000003e-07 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.644 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173282+0000", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.27e-06 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173288+0000", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7409999999999998e-06 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173293+0000", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.459e-06 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.645 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173337+0000", 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4002999999999999e-05 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173476+0000", 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000139665 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 1", 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.173732+0000", 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000256056 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.174787+0000", 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0010545419999999999 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213911+0000", 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039123915000000002 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213977+0000", 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.6469000000000003e-05 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.647 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.214023+0000", 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6109000000000002e-05 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23275 3.1c e55/54 3:3fef9174:::rbd_data.118227bb2f9.0000000000000000:head v 55'158, mlcod=55'154)", 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.35989423999999, 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.097924550999999999, 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.648 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213870+0000", 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213871+0000", 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.4400000000000001e-07 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.649 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213877+0000", 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0050000000000001e-06 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213882+0000", 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7699999999999998e-06 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.213886+0000", 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8809999999999998e-06 2024-05-08T03:22:12.650 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253556+0000", 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.039669395000000003 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.253575+0000", 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.9389e-05 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311762+0000", 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.058186999000000003 2024-05-08T03:22:12.651 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.311795+0000", 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.2768000000000002e-05 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:23277 3.d e55/35 3:b01af6ce:::rbd_data.118227bb2f9.0000000000000002:head v 55'71, mlcod=39'67)", 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 132.347120343, 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.098645508000000007, 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.652 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226644+0000", 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.653 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226645+0000", 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.3600000000000003e-07 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226651+0000", 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7200000000000003e-06 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226657+0000", 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4629999999999998e-06 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.654 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.226662+0000", 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7879999999999997e-06 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306254+0000", 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.079592349000000007 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.306271+0000", 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.6674999999999999e-05 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.655 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325253+0000", 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.018982056000000001 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:00.325290+0000", 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6721000000000003e-05 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.656 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:58300 3.1c e70/54 3:3e153a3c:::rbd_data.11827237cb53.0000000000000000:head v 70'363, mlcod=70'361)", 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 113.086381409, 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.023071852, 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.657 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487383+0000", 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.487385+0000", 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.497e-06 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495380+0000", 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0079952559999999992 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.658 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495406+0000", 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5910000000000001e-05 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495415+0000", 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1749999999999994e-06 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495499+0000", 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3647999999999996e-05 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.495519+0000", 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0513000000000002e-05 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510401+0000", 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.014881985 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:20:19.510455+0000", 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3868e-05 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126172 3.c e114/99 3:36cd0ac7:::rbd_data.1182d2294c83.0000000000000003:head v 114'675, mlcod=112'673)", 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 57.72807701, 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025413121, 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845688+0000", 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.845689+0000", 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2619999999999999e-06 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851573+0000", 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0058842139999999996 2024-05-08T03:22:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851586+0000", 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2839e-05 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851594+0000", 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.994e-06 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851664+0000", 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.046e-05 2024-05-08T03:22:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851682+0000", 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.7249e-05 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871062+0000", 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.019380755999999999 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.871101+0000", 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8346999999999997e-05 2024-05-08T03:22:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_repop(client.4482.0:126170 3.3 e114/87 3:c2464fb4:::rbd_data.1182d2294c83.0000000000000001:head v 114'676, mlcod=112'674)", 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 57.726458272999999, 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.025714192, 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847306+0000", 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.847308+0000", 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.3459999999999999e-06 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851483+0000", 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0041751130000000003 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851508+0000", 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.5496000000000001e-05 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851520+0000", 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2187999999999999e-05 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851590+0000", 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9750000000000001e-05 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.851616+0000", 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6084e-05 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.872972+0000", 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.021356209000000001 2024-05-08T03:22:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:14.873020+0000", 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8006000000000001e-05 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:12.669 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:12.673 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:22:12.674 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:12.670+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:12.772 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:12.770+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:12.819 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:12.819 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:12.819 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:12.819 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:12.829 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:22:12.836 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:22:12.837 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-05-08T03:22:12.837 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-05-08T03:22:12.913 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:12.911+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:12.973 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:12.971+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:12.987 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:12.987 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:12.987 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:12.987 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:12.987 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:12.996 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:22:13.081 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:13.079+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:13.145 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:13.145 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:13.145 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:13.145 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [ 2024-05-08T03:22:13.145 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:3836 3.14 3:2bb4acb3:::rbd_id.image_client.0-clone5:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 161.26810946699999, 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.128520475, 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 3836 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.146 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874132+0000", 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.147 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874134+0000", 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.141e-06 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874138+0000", 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.228e-06 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874148+0000", 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6479999999999998e-06 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.148 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874153+0000", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0880000000000002e-06 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874198+0000", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5111999999999998e-05 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:31.874204+0000", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5420000000000002e-06 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001079+0000", 2024-05-08T03:22:13.149 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1268746620000001 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001263+0000", 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00018362900000000001 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 6", 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.001514+0000", 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00025192900000000002 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002500+0000", 2024-05-08T03:22:13.150 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00098508899999999997 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002546+0000", 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.6913999999999998e-05 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002585+0000", 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.8554999999999999e-05 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.002653+0000", 2024-05-08T03:22:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7937999999999999e-05 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5009 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.926088695, 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859196999999999, 2024-05-08T03:22:13.152 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5009 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:13.153 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216153+0000", 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.154 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216154+0000", 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.9399999999999993e-07 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216157+0000", 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9550000000000001e-06 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216160+0000", 2024-05-08T03:22:13.155 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4779999999999998e-06 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247853+0000", 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031692691000000002 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.247922+0000", 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.9086999999999998e-05 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.156 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248012+0000", 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0789000000000001e-05 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5010 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92607917399999, 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031958926999999998, 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.157 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5010 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216163+0000", 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.05e-07 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216164+0000", 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8200000000000008e-07 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.159 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216167+0000", 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.802e-06 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216169+0000", 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4509999999999999e-06 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248028+0000", 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031859197999999998 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248067+0000", 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.841e-05 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248122+0000", 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4879000000000001e-05 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5011 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.161 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92606983300001, 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032146019999999997, 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5011 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.162 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216172+0000", 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.04e-07 2024-05-08T03:22:13.163 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216173+0000", 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.032e-06 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216176+0000", 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.773e-06 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216179+0000", 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.407e-06 2024-05-08T03:22:13.164 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248154+0000", 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.031975669999999998 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248226+0000", 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1977000000000006e-05 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248318+0000", 2024-05-08T03:22:13.165 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.1957000000000001e-05 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5012 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92606061800001, 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032249292999999998, 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.166 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5012 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216181+0000", 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.167 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216182+0000", 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216183+0000", 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1519999999999999e-06 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216185+0000", 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.7970000000000001e-06 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.168 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216188+0000", 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4830000000000002e-06 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248332+0000", 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032143722 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248375+0000", 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.2996e-05 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.169 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248431+0000", 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.5940000000000003e-05 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5013 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92605125599999, 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032431940999999999, 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.170 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5013 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.171 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216191+0000", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216192+0000", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.6200000000000006e-07 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216195+0000", 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.751e-06 2024-05-08T03:22:13.172 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216197+0000", 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.419e-06 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248464+0000", 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032266851999999999 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248534+0000", 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0301000000000003e-05 2024-05-08T03:22:13.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248623+0000", 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.8436000000000005e-05 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5014 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92604182400001, 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032536558, 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.174 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5014 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.175 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216200+0000", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1799999999999999e-07 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216201+0000", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.012e-06 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216204+0000", 2024-05-08T03:22:13.176 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9270000000000001e-06 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216207+0000", 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.553e-06 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248650+0000", 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.03244267 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.177 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248687+0000", 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7316e-05 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248737+0000", 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9861999999999998e-05 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5015 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.178 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.926032408, 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032655038999999997, 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5015 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.179 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216210+0000", 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1299999999999999e-07 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216211+0000", 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9899999999999999e-07 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.180 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216213+0000", 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.6809999999999998e-06 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216216+0000", 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3779999999999999e-06 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248766+0000", 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032550229999999999 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248810+0000", 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.3778e-05 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248865+0000", 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.486e-05 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.182 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5016 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92602339000001, 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032761930000000002, 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5016 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.183 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216219+0000", 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.03e-07 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.184 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216220+0000", 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.6300000000000004e-07 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216222+0000", 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8040000000000002e-06 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216225+0000", 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4260000000000002e-06 2024-05-08T03:22:13.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248886+0000", 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032661018 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248926+0000", 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.0485999999999998e-05 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.186 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.248980+0000", 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.4129999999999998e-05 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5017 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92601434700001, 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032861836999999998, 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.187 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5017 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.188 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216228+0000", 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1899999999999999e-07 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216231+0000", 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.5130000000000001e-06 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216234+0000", 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.999e-06 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216237+0000", 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4990000000000001e-06 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249003+0000", 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032766427000000001 2024-05-08T03:22:13.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249042+0000", 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.9178999999999999e-05 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249089+0000", 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.7000999999999999e-05 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.191 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5018 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92600205400001, 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033010322000000002, 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5018 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.192 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216240+0000", 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0900000000000001e-07 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.193 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216241+0000", 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0559999999999999e-06 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216244+0000", 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8830000000000002e-06 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216246+0000", 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.4080000000000001e-06 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.194 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249111+0000", 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.032864314999999998 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249179+0000", 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.8617999999999999e-05 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249250+0000", 2024-05-08T03:22:13.195 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.0833000000000004e-05 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5019 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92599278700001, 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033159319, 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.196 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5019 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.197 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216249+0000", 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216254+0000", 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0900000000000004e-06 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.198 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216258+0000", 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.1659999999999998e-06 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216260+0000", 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3649999999999998e-06 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249282+0000", 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033021517 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.199 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249341+0000", 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.9567000000000001e-05 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249408+0000", 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.7408000000000002e-05 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.200 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5020 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.925978929, 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033281855999999999, 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5020 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.201 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216263+0000", 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.1500000000000001e-07 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.202 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216264+0000", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.8700000000000004e-07 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216267+0000", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8880000000000001e-06 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216270+0000", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.452e-06 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249439+0000", 2024-05-08T03:22:13.203 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033169570000000002 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249488+0000", 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.8677000000000002e-05 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249545+0000", 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.7067000000000001e-05 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.204 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:5021 3.e 3:7695e779:::rbd_data.1182b3dbc99f.0000000000000000:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 159.92596963599999, 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033419810000000001, 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 5021 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.205 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216272+0000", 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216273+0000", 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.3200000000000001e-07 2024-05-08T03:22:13.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216274+0000", 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.4799999999999997e-07 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216276+0000", 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9859999999999999e-06 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.207 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.216279+0000", 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.435e-06 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249570+0000", 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.033291302000000002 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249615+0000", 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.4678999999999997e-05 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.208 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:19:33.249692+0000", 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.7228e-05 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:144875 3.14 3.e36d51d4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:59.785375+0000", 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.356866712, 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.00537221, 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "reached pg", 2024-05-08T03:22:13.209 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 144875 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785375+0000", 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785375+0000", 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785377+0000", 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.2110000000000001e-06 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785386+0000", 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.2230000000000007e-06 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785414+0000", 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.8807e-05 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.211 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.785427+0000", 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2452999999999999e-05 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.788579+0000", 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.003151839 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.788590+0000", 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1430999999999999e-05 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.212 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790727+0000", 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.002136973 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790747+0000", 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.0273000000000001e-05 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:144876 3.14 3.e36d51d4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-05-08T03:22:13.213 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:21:59.791087+0000", 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 13.351154737, 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99971595199999996, 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "reached pg", 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 144876 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.214 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791087+0000", 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791087+0000", 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791089+0000", 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5889999999999999e-06 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.215 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791094+0000", 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.0649999999999998e-06 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791104+0000", 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.0071000000000001e-05 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791111+0000", 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.4909999999999999e-06 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.216 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791312+0000", 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00020073000000000001 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:21:59.791320+0000", 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3110000000000006e-06 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790794+0000", 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99947371699999998 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:00.790803+0000", 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9779999999999994e-06 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:145167 3.15 3:adc0bd6f:::rbd_header.1182ca0d3624:head [watch reconnect cookie 140222361533216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:03.811167+0000", 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.3310748869999998, 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99499526199999999, 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.218 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 145167 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811167+0000", 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.219 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811167+0000", 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811169+0000", 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.5090000000000001e-06 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.220 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811175+0000", 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.1480000000000003e-06 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811231+0000", 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.6064999999999999e-05 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811261+0000", 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 2.9816999999999999e-05 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811798+0000", 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053767099999999998 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811807+0000", 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.3990000000000004e-06 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.799321+0000", 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.98751383599999998 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.222 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.799851+0000", 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00053066799999999998 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806162+0000", 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0063111490000000003 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:145168 3.15 3:adc0bd6f:::rbd_header.1182ca0d3624:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:13.223 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:03.811587+0000", 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.330654891, 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.99473379799999995, 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 145168 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.224 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811587+0000", 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811587+0000", 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811588+0000", 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.6199999999999997e-07 2024-05-08T03:22:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811592+0000", 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.5279999999999997e-06 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811601+0000", 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.4810000000000006e-06 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811608+0000", 2024-05-08T03:22:13.226 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.063e-06 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811831+0000", 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000222644 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:03.811836+0000", 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8629999999999996e-06 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.227 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.800614+0000", 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.98877804599999997 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.800704+0000", 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 8.9773999999999998e-05 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for subops from 4", 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.800944+0000", 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000239418 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.228 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "op_commit", 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.805347+0000", 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0044031679999999998 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806203+0000", 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.00085625000000000002 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "commit_sent", 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806267+0000", 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.4183000000000005e-05 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.229 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806321+0000", 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3618000000000001e-05 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4482.0:145169 3.15 3:adc0bd6f:::rbd_header.1182ca0d3624:head [watch ping cookie 140222361533216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:04.078332+0000", 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 9.0639096610000003, 2024-05-08T03:22:13.230 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.72815680000000005, 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "started", 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4482", 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 145169 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078332+0000", 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.231 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078332+0000", 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078333+0000", 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.1710000000000001e-06 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.232 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078338+0000", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.357e-06 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078351+0000", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 1.2685e-05 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078358+0000", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.1339999999999998e-06 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.233 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078412+0000", 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.3999999999999998e-05 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.078421+0000", 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 9.0149999999999992e-06 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.801238+0000", 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.72281739099999998 2024-05-08T03:22:13.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for rw locks", 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.801311+0000", 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.2501000000000003e-05 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806349+0000", 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.0050382530000000003 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "started", 2024-05-08T03:22:13.235 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806409+0000", 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.0676e-05 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.806489+0000", 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 7.9616999999999993e-05 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "description": "osd_op(client.4175.0:12 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163)", 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "initiated_at": "2024-05-08T03:22:04.210739+0000", 2024-05-08T03:22:13.236 INFO:teuthology.orchestra.run.smithi188.stdout: "age": 8.9315029480000003, 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.58313978099999997, 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "continuous": false, 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "type_data": { 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "flag_point": "reached pg", 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "client_info": { 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "client": "client.4175", 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "client_addr": "172.21.15.177:0/3444697585", 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "tid": 12 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "events": [ 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "initiated", 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210739+0000", 2024-05-08T03:22:13.237 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "header_read", 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210739+0000", 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "throttled", 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210740+0000", 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 4.9800000000000004e-07 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.238 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "all_read", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210743+0000", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.3380000000000001e-06 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "dispatched", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210749+0000", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 6.5200000000000003e-06 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "queued_for_pg", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210753+0000", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.7179999999999998e-06 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.239 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210790+0000", 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 3.6514000000000003e-05 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "waiting for peered", 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.210795+0000", 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 5.8000000000000004e-06 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "reached_pg", 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.793613+0000", 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.58281794499999995 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: { 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "event": "done", 2024-05-08T03:22:13.240 INFO:teuthology.orchestra.run.smithi188.stdout: "time": "2024-05-08T03:22:04.793879+0000", 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 0.000265448 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-08T03:22:13.241 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:13.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:22:13.243 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:13.241+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:13.343 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:13.341+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:13.389 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:13.390 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:13.390 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:13.390 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:13.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:22:13.443 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:13.443+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:13.544 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:13.544 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:13.544 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:13.544 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:13.544 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:13.544 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:13.542+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:13.552 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:22:13.645 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:13.644+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:13.697 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:13.697 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:13.697 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:13.697 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:22:13.697 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:13.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T03:22:13.746 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:13.744+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:13.815 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:13.815 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:13.815 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:13.815 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:13.824 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T03:22:13.832 INFO:teuthology.orchestra.run.smithi177.stderr:osd set-full-ratio 0.95 2024-05-08T03:22:13.837 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:22:13.846 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:13.846+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:13.945 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:13.945 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:13.945 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:13.945 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:13.945 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:13.947 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:13.945+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:13.955 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T03:22:14.048 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:14.048+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:14.062 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:14.062 INFO:teuthology.orchestra.run.smithi177.stdout: "size": 20, 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 600, 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [ 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1641 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.86051751599999, 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017556473999999999, 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1641 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.063 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199717+0000", 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.1199999999999999e-07 2024-05-08T03:22:14.064 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199718+0000", 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.5799999999999998e-07 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199721+0000", 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.892e-06 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199723+0000", 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3800000000000001e-06 2024-05-08T03:22:14.065 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217174+0000", 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017450911999999999 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217217+0000", 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3028999999999997e-05 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217273+0000", 2024-05-08T03:22:14.066 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6091000000000002e-05 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1642 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.86050806599999, 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017673867999999999, 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1642 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.068 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199726+0000", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8200000000000001e-07 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199727+0000", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1829999999999999e-06 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199731+0000", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.8349999999999997e-06 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199734+0000", 2024-05-08T03:22:14.069 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.402e-06 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217301+0000", 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017567580999999999 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217345+0000", 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3331999999999998e-05 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.070 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217400+0000", 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.5253000000000001e-05 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1643 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.86049532999999, 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017792624999999999, 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.071 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1643 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.072 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199739+0000", 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4699999999999998e-07 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199740+0000", 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.029e-06 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199743+0000", 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.163e-06 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.073 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199746+0000", 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4279999999999999e-06 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217434+0000", 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017688820000000001 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217478+0000", 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3451999999999997e-05 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.074 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217531+0000", 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3486000000000003e-05 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1644 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.860484754, 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017906828999999999, 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.075 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1644 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.076 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199749+0000", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0800000000000001e-07 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199750+0000", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.0130000000000001e-06 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199753+0000", 2024-05-08T03:22:14.077 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9790000000000002e-06 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199756+0000", 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2510000000000001e-06 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217558+0000", 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017802512999999999 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217603+0000", 2024-05-08T03:22:14.078 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4467e-05 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217656+0000", 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3397999999999999e-05 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1645 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.86047536699999, 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018026106, 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.079 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1645 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.080 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199759+0000", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.03e-07 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199760+0000", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9100000000000002e-07 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199763+0000", 2024-05-08T03:22:14.081 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9510000000000002e-06 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199765+0000", 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3250000000000002e-06 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217691+0000", 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925863 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217731+0000", 2024-05-08T03:22:14.082 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0107000000000001e-05 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217785+0000", 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3766e-05 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:1646 3.12 3:498a4cf5:::rbd_data.1182b3dbc99f.0000000000000007:4 [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.86046618699999, 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018136026999999999, 2024-05-08T03:22:14.083 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 1646 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.084 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199768+0000", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.23e-07 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199769+0000", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.8199999999999998e-07 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.085 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199772+0000", 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9699999999999999e-06 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.199774+0000", 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3199999999999998e-06 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217808+0000", 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018033961000000001 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.086 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217851+0000", 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2460999999999999e-05 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.217904+0000", 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.321e-05 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2031 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:22:14.087 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.629641152, 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017634125, 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2031 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.088 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430593+0000", 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.13e-07 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430596+0000", 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.244e-06 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.089 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430603+0000", 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.9040000000000003e-06 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430607+0000", 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5879999999999999e-06 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448142+0000", 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017534999999999999 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.090 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448178+0000", 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.5964000000000003e-05 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448227+0000", 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.8912000000000002e-05 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2032 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.091 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.62960891500001, 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017726393999999999, 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2032 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:22:14.092 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430625+0000", 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.03e-07 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430627+0000", 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.517e-06 2024-05-08T03:22:14.093 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430632+0000", 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7609999999999998e-06 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430635+0000", 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1209999999999998e-06 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448260+0000", 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017625306 2024-05-08T03:22:14.094 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448300+0000", 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0108999999999998e-05 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448351+0000", 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1276999999999997e-05 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2033 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.095 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.629595585, 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017813895, 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2033 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:22:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430638+0000", 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430639+0000", 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.0599999999999999e-07 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430640+0000", 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1370000000000001e-06 2024-05-08T03:22:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430647+0000", 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1489999999999997e-06 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430650+0000", 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.7800000000000001e-06 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448373+0000", 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017723182000000001 2024-05-08T03:22:14.098 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448408+0000", 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4869000000000001e-05 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448452+0000", 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4571999999999999e-05 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2034 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.099 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.62958061099999, 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017925831999999999, 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2034 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.100 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430653+0000", 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430654+0000", 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.0600000000000001e-07 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430655+0000", 2024-05-08T03:22:14.101 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.26e-06 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430659+0000", 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.0929999999999996e-06 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430662+0000", 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.6070000000000002e-06 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448484+0000", 2024-05-08T03:22:14.102 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017822635 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448526+0000", 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.2119000000000001e-05 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448579+0000", 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.2812000000000002e-05 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.103 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2035 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.62956930199999, 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018018925000000002, 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2035 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:22:14.104 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430665+0000", 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.22e-07 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430672+0000", 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.7789999999999998e-06 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.105 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430676+0000", 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4000000000000002e-06 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430679+0000", 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.858e-06 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448601+0000", 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017922376 2024-05-08T03:22:14.106 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448636+0000", 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4368e-05 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448684+0000", 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7922e-05 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2036 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.107 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.62955163300001, 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018122243, 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2036 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.108 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430682+0000", 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430683+0000", 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5100000000000001e-07 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430687+0000", 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1080000000000003e-06 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.109 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430691+0000", 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4730000000000002e-06 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430694+0000", 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8849999999999999e-06 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448716+0000", 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018021486 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.110 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448755+0000", 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9141999999999999e-05 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448805+0000", 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.9898000000000002e-05 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2037 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.62953643500001, 2024-05-08T03:22:14.111 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018209809, 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2037 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.112 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430698+0000", 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.2000000000000001e-07 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430699+0000", 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.099e-06 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.113 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430705+0000", 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 6.4740000000000002e-06 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430708+0000", 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.8760000000000001e-06 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448825+0000", 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018117202999999998 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.114 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448860+0000", 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4134000000000002e-05 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448907+0000", 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7803000000000003e-05 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:2038 3.1b 3:dc2c3b18:::rbd_data.1182b3dbc99f.0000000000000002:4 [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 165.62952234100001, 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018314861000000002, 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.115 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 2038 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.116 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430712+0000", 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.4400000000000001e-07 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430713+0000", 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1510000000000001e-06 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430716+0000", 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1920000000000001e-06 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.117 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.430719+0000", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3489999999999999e-06 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448932+0000", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018213213999999998 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.448975+0000", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3305e-05 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.118 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:28.449027+0000", 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1406000000000002e-05 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12443 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14680064~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 157.854568368, 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.036968674999999999, 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12443 2024-05-08T03:22:14.119 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205666+0000", 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.205668+0000", 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.3910000000000001e-06 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.120 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207650+0000", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.0019824460000000001 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207668+0000", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.7125999999999998e-05 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207677+0000", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.7450000000000007e-06 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207749+0000", 2024-05-08T03:22:14.121 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.2038000000000004e-05 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.207893+0000", 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00014315399999999999 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.208804+0000", 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00091170499999999996 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.225405+0000", 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.016600255000000001 2024-05-08T03:22:14.122 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242556+0000", 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017151650000000001 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242603+0000", 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.7036999999999997e-05 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:36.242634+0000", 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.1127999999999997e-05 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.123 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12933 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 786432~98304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 156.931674566, 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017936612000000001, 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12933 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:22:14.124 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128559+0000", 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128560+0000", 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.67e-07 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.125 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128561+0000", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.111e-06 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128564+0000", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.021e-06 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128568+0000", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.7799999999999998e-06 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145823+0000", 2024-05-08T03:22:14.126 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017254915999999999 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.145912+0000", 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.9714000000000002e-05 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146496+0000", 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00058380300000000003 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.127 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12934 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 156.93166107799999, 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018500744999999999, 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12934 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.128 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128573+0000", 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.9999999999999999e-07 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.129 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128575+0000", 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.852e-06 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128591+0000", 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.5835e-05 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128595+0000", 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.4710000000000001e-06 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.130 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146528+0000", 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.017932527 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.146600+0000", 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 7.1971000000000006e-05 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147074+0000", 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00047378900000000001 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.131 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12949 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 156.931632097, 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019072189, 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12949 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.132 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128602+0000", 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.1699999999999999e-07 2024-05-08T03:22:14.133 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128605+0000", 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.379e-06 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128616+0000", 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1314e-05 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128621+0000", 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.3249999999999998e-06 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.134 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147126+0000", 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.018504487 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147183+0000", 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6807e-05 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147674+0000", 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00049145999999999999 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.135 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:12950 3.1f 3:ffc390a7:::rbd_data.1182ca0d3624.0000000000000001:a [sparse-read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 156.931605491, 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019491841999999999, 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "started", 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 12950 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.136 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128629+0000", 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.3099999999999998e-07 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.137 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128632+0000", 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 2.5610000000000001e-06 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128641+0000", 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.0070000000000003e-06 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.128646+0000", 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.4330000000000003e-06 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.138 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147702+0000", 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.019055595000000002 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.147784+0000", 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 8.2272000000000001e-05 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.148120+0000", 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00033654300000000002 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.139 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "description": "osd_op(client.4482.0:13255 3.17 3:eb86d788:::rbd_data.118231ecabb9.000000000000003a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "initiated_at": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "age": 156.49885723, 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.025242460000000001, 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "continuous": false, 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "type_data": { 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client_info": { 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client": "client.4482", 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "client_addr": "172.21.15.177:0/3826447822", 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "tid": 13255 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.140 INFO:teuthology.orchestra.run.smithi177.stdout: "events": [ 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "initiated", 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "header_read", 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561377+0000", 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "throttled", 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561378+0000", 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 9.6899999999999996e-07 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.141 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "all_read", 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561383+0000", 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.1950000000000002e-06 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "dispatched", 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561394+0000", 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 1.1306e-05 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "queued_for_pg", 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561400+0000", 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 5.6099999999999997e-06 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.142 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "reached_pg", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561447+0000", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.6649e-05 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "started", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.561577+0000", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.00013032899999999999 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "waiting for subops from 5", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.565970+0000", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.004393124 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "op_commit", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586500+0000", 2024-05-08T03:22:14.143 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 0.020529556000000001 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "sub_op_commit_rec", 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586545+0000", 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 4.5856000000000003e-05 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "commit_sent", 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586585+0000", 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.9801000000000001e-05 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: { 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "event": "done", 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "time": "2024-05-08T03:19:37.586619+0000", 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: "duration": 3.4065e-05 2024-05-08T03:22:14.144 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.145 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.145 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.145 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-05-08T03:22:14.145 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-05-08T03:22:14.145 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:14.146 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T03:22:14.187 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:14.187+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:14.250 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:14.248+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:14.293 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:14.294 INFO:teuthology.orchestra.run.smithi177.stdout: "num_ops": 0 2024-05-08T03:22:14.294 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:14.302 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T03:22:14.313 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T03:22:14.389 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:14.388+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:14.409 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:14.409 INFO:teuthology.orchestra.run.smithi177.stdout: "ops": [], 2024-05-08T03:22:14.409 INFO:teuthology.orchestra.run.smithi177.stdout: "complaint_time": 30, 2024-05-08T03:22:14.409 INFO:teuthology.orchestra.run.smithi177.stdout: "num_blocked_ops": 0 2024-05-08T03:22:14.409 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:14.419 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T03:22:14.451 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:14.449+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:14.468 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:14.469 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:14.487 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:14.487 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:14.504 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:14.504 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:14.521 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:14.521 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:14.538 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:14.539 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:14.552 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:14.551+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:14.556 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:14.556 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:14.566 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:14.566 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:14.573 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:14.573 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:14.576 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T03:22:14.590 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:14.591 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:14.653 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:14.652+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:14.694 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:14.695 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:14.704 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T03:22:14.754 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:14.753+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:14.814 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:14.814 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:14.824 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T03:22:14.855 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:14.853+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:14.956 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:14.955+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:14.972 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:14.972 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:14.981 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T03:22:15.057 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:15.056+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:15.126 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.126 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.135 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T03:22:15.178 INFO:tasks.ceph.osd.2.smithi177.stderr:2024-05-08T03:22:15.178+0000 7f9deb768640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32475) UID: 0 2024-05-08T03:22:15.240 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.240 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.249 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T03:22:15.291 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:15.289+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:15.360 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:15.359+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:15.397 INFO:teuthology.orchestra.run.smithi177.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.397 INFO:teuthology.orchestra.run.smithi177.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.406 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T03:22:15.462 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:15.460+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:15.513 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.513 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.522 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T03:22:15.563 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:15.561+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:15.663 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-08T03:22:15.661+0000 7fbf30cab640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44993) UID: 0 2024-05-08T03:22:15.667 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.667 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T03:22:15.764 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:15.762+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:15.822 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.822 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.831 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T03:22:15.860 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:22:15.903 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:15.902+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:15.966 INFO:tasks.ceph.osd.0.smithi177.stderr:2024-05-08T03:22:15.964+0000 7f7603a26640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32461) UID: 0 2024-05-08T03:22:15.985 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:15.985 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:15.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T03:22:16.067 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:16.066+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:16.100 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:16.100 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:16.108 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T03:22:16.168 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-08T03:22:16.166+0000 7f94cd2fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32753) UID: 0 2024-05-08T03:22:16.214 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:16.214 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:16.223 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T03:22:16.307 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:16.305+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:16.368 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:16.368 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:16.370 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:16.369+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:16.376 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T03:22:16.471 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:16.469+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:16.580 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:16.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T03:22:16.587 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-08T03:22:16.581+0000 7f3d78ec0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32744) UID: 0 2024-05-08T03:22:16.602 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T03:22:16.685 INFO:tasks.ceph.osd.1.smithi177.stderr:2024-05-08T03:22:16.683+0000 7fbe734c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32469) UID: 0 2024-05-08T03:22:16.731 INFO:teuthology.orchestra.run.smithi177.stdout:osd.0: {} 2024-05-08T03:22:16.732 INFO:teuthology.orchestra.run.smithi177.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T03:22:16.733 INFO:teuthology.orchestra.run.smithi188.stdout:ERROR: (22) Invalid argument 2024-05-08T03:22:16.733 INFO:teuthology.orchestra.run.smithi188.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T03:22:16.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T03:22:16.748 INFO:teuthology.orchestra.run.smithi177.stdout:osd.1: {} 2024-05-08T03:22:16.748 INFO:teuthology.orchestra.run.smithi177.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T03:22:16.765 INFO:teuthology.orchestra.run.smithi177.stdout:osd.2: {} 2024-05-08T03:22:16.765 INFO:teuthology.orchestra.run.smithi177.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T03:22:16.781 INFO:teuthology.orchestra.run.smithi177.stdout:osd.3: {} 2024-05-08T03:22:16.782 INFO:teuthology.orchestra.run.smithi177.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T03:22:16.785 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:16.783+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:16.798 INFO:teuthology.orchestra.run.smithi177.stdout:osd.4: {} 2024-05-08T03:22:16.798 INFO:teuthology.orchestra.run.smithi177.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T03:22:16.817 INFO:teuthology.orchestra.run.smithi177.stdout:osd.5: {} 2024-05-08T03:22:16.817 INFO:teuthology.orchestra.run.smithi177.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T03:22:16.835 INFO:teuthology.orchestra.run.smithi177.stdout:osd.6: {} 2024-05-08T03:22:16.836 INFO:teuthology.orchestra.run.smithi177.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T03:22:16.850 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:16.850 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:16.850 INFO:teuthology.orchestra.run.smithi188.stdout: "num_ops": 0 2024-05-08T03:22:16.850 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:16.853 INFO:teuthology.orchestra.run.smithi177.stdout:osd.7: {} 2024-05-08T03:22:16.853 INFO:teuthology.orchestra.run.smithi177.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T03:22:16.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T03:22:16.860 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:22:16.917 INFO:tasks.ceph.osd.3.smithi177.stderr:2024-05-08T03:22:16.916+0000 7fce6df7b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32483) UID: 0 2024-05-08T03:22:16.962 INFO:teuthology.orchestra.run.smithi177.stdout:All operations completed A-OK! 2024-05-08T03:22:16.977 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:16.977 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [], 2024-05-08T03:22:16.977 INFO:teuthology.orchestra.run.smithi188.stdout: "complaint_time": 30, 2024-05-08T03:22:16.977 INFO:teuthology.orchestra.run.smithi188.stdout: "num_blocked_ops": 0 2024-05-08T03:22:16.978 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:16.987 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T03:22:16.987 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-08T03:22:16.985+0000 7fd6dd14d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32748) UID: 0 2024-05-08T03:22:17.067 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:17.067 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-05-08T03:22:17.077 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-05-08T03:22:17.101 INFO:tasks.thrashosds:joining thrashosds 2024-05-08T03:22:17.132 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-08T03:22:17.132 INFO:teuthology.orchestra.run.smithi188.stdout: "size": 20, 2024-05-08T03:22:17.133 INFO:teuthology.orchestra.run.smithi188.stdout: "duration": 600, 2024-05-08T03:22:17.133 INFO:teuthology.orchestra.run.smithi188.stdout: "ops": [] 2024-05-08T03:22:17.133 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-08T03:22:18.853 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-08T03:22:18.853 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-05-08T03:22:18.854 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-05-08T03:22:18.876 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T03:22:19.177 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:22:19.175+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e176 definitely_dead 0 2024-05-08T03:22:19.787 INFO:teuthology.orchestra.run.smithi177.stderr:marked in osd.5. 2024-05-08T03:22:19.787 INFO:teuthology.orchestra.run.smithi177.stderr:noscrub is unset 2024-05-08T03:22:19.806 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-05-08T03:22:19.806 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-08T03:22:19.806 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-05-08T03:22:19.806 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-05-08T03:22:19.807 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T03:22:20.175 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:22:20.174+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e177 definitely_dead 0 2024-05-08T03:22:20.793 INFO:teuthology.orchestra.run.smithi177.stderr:nodeep-scrub is unset 2024-05-08T03:22:20.793 INFO:teuthology.orchestra.run.smithi177.stderr:marked in osd.3. 2024-05-08T03:22:20.812 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-05-08T03:22:20.812 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-08T03:22:20.812 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-05-08T03:22:20.812 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-05-08T03:22:21.149 INFO:tasks.ceph.mon.a.smithi177.stderr:2024-05-08T03:22:21.147+0000 7fc7dc33c640 -1 mon.a@0(leader).osd e178 definitely_dead 0 2024-05-08T03:22:21.802 INFO:teuthology.orchestra.run.smithi177.stderr:marked in osd.0. 2024-05-08T03:22:21.823 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-05-08T03:22:21.823 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-05-08T03:22:21.954 INFO:teuthology.orchestra.run.smithi177.stdout:mon.a: {} 2024-05-08T03:22:21.955 INFO:teuthology.orchestra.run.smithi177.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T03:22:21.963 INFO:teuthology.orchestra.run.smithi177.stdout:mon.b: {} 2024-05-08T03:22:21.963 INFO:teuthology.orchestra.run.smithi177.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T03:22:21.972 INFO:teuthology.orchestra.run.smithi177.stdout:mon.c: {} 2024-05-08T03:22:21.973 INFO:teuthology.orchestra.run.smithi177.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T03:22:21.984 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-08T03:22:22.791 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.0 to 1 (10000) 2024-05-08T03:22:22.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-08T03:22:23.792 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-08T03:22:23.810 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T03:22:24.794 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T03:22:24.812 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-05-08T03:22:25.796 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-05-08T03:22:25.817 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T03:22:26.796 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T03:22:26.824 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-08T03:22:27.794 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-08T03:22:27.812 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-08T03:22:28.796 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.3 to 1 (10000) 2024-05-08T03:22:28.815 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-08T03:22:29.794 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-08T03:22:29.811 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T03:22:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T03:22:30.937 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-08T03:22:31.918 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-08T03:22:31.934 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-08T03:22:32.924 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.6 to 1 (10000) 2024-05-08T03:22:32.942 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-05-08T03:22:33.933 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-05-08T03:22:33.947 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-08T03:22:34.935 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.7 to 1 (10000) 2024-05-08T03:22:34.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-05-08T03:22:35.945 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-05-08T03:22:35.961 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-08T03:22:36.950 INFO:teuthology.orchestra.run.smithi177.stderr:reweighted osd.5 to 1 (10000) 2024-05-08T03:22:36.965 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-08T03:22:37.952 INFO:teuthology.orchestra.run.smithi177.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-08T03:22:37.968 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-05-08T03:22:37.968 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-05-08T03:22:37.968 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-05-08T03:22:37.968 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-05-08T03:22:37.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T03:22:37.969 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T03:22:38.261 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:38.261 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":195,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:22:37.950144+0000","last_up_change":"2024-05-08T03:21:03.827528+0000","last_in_change":"2024-05-08T03:22:21.787702+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T03:19:11.648360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T03:19:12.206654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T03:19:19.077332+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":49,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6809","nonce":2599410006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6811","nonce":2599410006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6815","nonce":2599410006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6813","nonce":2599410006}]},"public_addr":"172.21.15.177:6809/2599410006","cluster_addr":"172.21.15.177:6811/2599410006","heartbeat_back_addr":"172.21.15.177:6815/2599410006","heartbeat_front_addr":"172.21.15.177:6813/2599410006","state":["exists","up"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6823","nonce":3471641477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6827","nonce":3471641477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6831","nonce":3471641477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6829","nonce":3471641477}]},"public_addr":"172.21.15.177:6823/3471641477","cluster_addr":"172.21.15.177:6827/3471641477","heartbeat_back_addr":"172.21.15.177:6831/3471641477","heartbeat_front_addr":"172.21.15.177:6829/3471641477","state":["exists","up"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6801","nonce":2694847262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6803","nonce":2694847262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6807","nonce":2694847262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6805","nonce":2694847262}]},"public_addr":"172.21.15.177:6801/2694847262","cluster_addr":"172.21.15.177:6803/2694847262","heartbeat_back_addr":"172.21.15.177:6807/2694847262","heartbeat_front_addr":"172.21.15.177:6805/2694847262","state":["exists","up"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6817","nonce":4093771454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6819","nonce":4093771454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6826","nonce":4093771454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6822","nonce":4093771454}]},"public_addr":"172.21.15.177:6817/4093771454","cluster_addr":"172.21.15.177:6819/4093771454","heartbeat_back_addr":"172.21.15.177:6826/4093771454","heartbeat_front_addr":"172.21.15.177:6822/4093771454","state":["exists","up"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6801","nonce":1585905529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6803","nonce":1585905529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6807","nonce":1585905529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6805","nonce":1585905529}]},"public_addr":"172.21.15.188:6801/1585905529","cluster_addr":"172.21.15.188:6803/1585905529","heartbeat_back_addr":"172.21.15.188:6807/1585905529","heartbeat_front_addr":"172.21.15.188:6805/1585905529","state":["exists","up"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":58,"up_from":106,"up_thru":182,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6817","nonce":1469026711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6819","nonce":1469026711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6823","nonce":1469026711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6821","nonce":1469026711}]},"public_addr":"172.21.15.188:6817/1469026711","cluster_addr":"172.21.15.188:6819/1469026711","heartbeat_back_addr":"172.21.15.188:6823/1469026711","heartbeat_front_addr":"172.21.15.188:6821/1469026711","state":["exists","up"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6809","nonce":3787670496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6811","nonce":3787670496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6815","nonce":3787670496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6813","nonce":3787670496}]},"public_addr":"172.21.15.188:6809/3787670496","cluster_addr":"172.21.15.188:6811/3787670496","heartbeat_back_addr":"172.21.15.188:6815/3787670496","heartbeat_front_addr":"172.21.15.188:6813/3787670496","state":["exists","up"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6825","nonce":405203122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6827","nonce":405203122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6831","nonce":405203122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6829","nonce":405203122}]},"public_addr":"172.21.15.188:6825/405203122","cluster_addr":"172.21.15.188:6827/405203122","heartbeat_back_addr":"172.21.15.188:6831/405203122","heartbeat_front_addr":"172.21.15.188:6829/405203122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.148758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.170231+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.129002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.179213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.075940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122695+0000","dead_epoch":61},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:10.421746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122924+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.19","osds":[3,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:22:38.274 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T03:22:38.274 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T03:22:38.564 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:38.564 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":195,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:22:37.950144+0000","last_up_change":"2024-05-08T03:21:03.827528+0000","last_in_change":"2024-05-08T03:22:21.787702+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T03:19:11.648360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T03:19:12.206654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T03:19:19.077332+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":49,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6809","nonce":2599410006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6811","nonce":2599410006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6815","nonce":2599410006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6813","nonce":2599410006}]},"public_addr":"172.21.15.177:6809/2599410006","cluster_addr":"172.21.15.177:6811/2599410006","heartbeat_back_addr":"172.21.15.177:6815/2599410006","heartbeat_front_addr":"172.21.15.177:6813/2599410006","state":["exists","up"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6823","nonce":3471641477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6827","nonce":3471641477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6831","nonce":3471641477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6829","nonce":3471641477}]},"public_addr":"172.21.15.177:6823/3471641477","cluster_addr":"172.21.15.177:6827/3471641477","heartbeat_back_addr":"172.21.15.177:6831/3471641477","heartbeat_front_addr":"172.21.15.177:6829/3471641477","state":["exists","up"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6801","nonce":2694847262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6803","nonce":2694847262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6807","nonce":2694847262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6805","nonce":2694847262}]},"public_addr":"172.21.15.177:6801/2694847262","cluster_addr":"172.21.15.177:6803/2694847262","heartbeat_back_addr":"172.21.15.177:6807/2694847262","heartbeat_front_addr":"172.21.15.177:6805/2694847262","state":["exists","up"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6817","nonce":4093771454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6819","nonce":4093771454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6826","nonce":4093771454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6822","nonce":4093771454}]},"public_addr":"172.21.15.177:6817/4093771454","cluster_addr":"172.21.15.177:6819/4093771454","heartbeat_back_addr":"172.21.15.177:6826/4093771454","heartbeat_front_addr":"172.21.15.177:6822/4093771454","state":["exists","up"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6801","nonce":1585905529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6803","nonce":1585905529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6807","nonce":1585905529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6805","nonce":1585905529}]},"public_addr":"172.21.15.188:6801/1585905529","cluster_addr":"172.21.15.188:6803/1585905529","heartbeat_back_addr":"172.21.15.188:6807/1585905529","heartbeat_front_addr":"172.21.15.188:6805/1585905529","state":["exists","up"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":58,"up_from":106,"up_thru":182,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6817","nonce":1469026711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6819","nonce":1469026711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6823","nonce":1469026711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6821","nonce":1469026711}]},"public_addr":"172.21.15.188:6817/1469026711","cluster_addr":"172.21.15.188:6819/1469026711","heartbeat_back_addr":"172.21.15.188:6823/1469026711","heartbeat_front_addr":"172.21.15.188:6821/1469026711","state":["exists","up"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6809","nonce":3787670496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6811","nonce":3787670496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6815","nonce":3787670496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6813","nonce":3787670496}]},"public_addr":"172.21.15.188:6809/3787670496","cluster_addr":"172.21.15.188:6811/3787670496","heartbeat_back_addr":"172.21.15.188:6815/3787670496","heartbeat_front_addr":"172.21.15.188:6813/3787670496","state":["exists","up"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6825","nonce":405203122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6827","nonce":405203122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6831","nonce":405203122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6829","nonce":405203122}]},"public_addr":"172.21.15.188:6825/405203122","cluster_addr":"172.21.15.188:6827/405203122","heartbeat_back_addr":"172.21.15.188:6831/405203122","heartbeat_front_addr":"172.21.15.188:6829/405203122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.148758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.170231+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.129002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.179213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.075940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122695+0000","dead_epoch":61},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:10.421746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122924+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.19","osds":[3,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:22:38.577 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T03:22:38.578 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T03:22:38.578 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T03:22:38.579 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T03:22:38.579 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T03:22:38.580 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T03:22:38.580 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T03:22:38.581 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T03:22:38.818 INFO:teuthology.orchestra.run.smithi177.stdout:455266533397 2024-05-08T03:22:38.819 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T03:22:38.823 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:38.823 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T03:22:38.831 INFO:teuthology.orchestra.run.smithi177.stdout:60129542191 2024-05-08T03:22:38.831 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T03:22:38.832 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:38.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T03:22:38.833 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:38.834 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T03:22:38.834 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:38.835 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T03:22:38.836 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:38.836 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T03:22:38.837 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:38.837 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T03:22:39.393 INFO:teuthology.orchestra.run.smithi177.stdout:455266533396 2024-05-08T03:22:39.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 455266533397 got 455266533396 for osd.5 2024-05-08T03:22:39.418 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:39.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.1 2024-05-08T03:22:39.450 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:39.452 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:39.461 INFO:teuthology.orchestra.run.smithi177.stdout:60129542188 2024-05-08T03:22:39.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.3 2024-05-08T03:22:39.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542190 for osd.2 2024-05-08T03:22:39.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542188 for osd.7 2024-05-08T03:22:39.489 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:39.499 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:39.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.4 2024-05-08T03:22:39.513 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:39.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.0 2024-05-08T03:22:39.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.6 2024-05-08T03:22:40.419 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T03:22:40.439 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T03:22:40.467 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T03:22:40.468 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T03:22:40.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T03:22:40.507 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T03:22:40.515 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T03:22:40.527 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T03:22:40.982 INFO:teuthology.orchestra.run.smithi177.stdout:455266533398 2024-05-08T03:22:41.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 455266533397 got 455266533398 for osd.5 2024-05-08T03:22:41.007 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.036 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:41.051 INFO:teuthology.orchestra.run.smithi177.stdout:60129542191 2024-05-08T03:22:41.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.1 2024-05-08T03:22:41.056 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542191 for osd.2 2024-05-08T03:22:41.067 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.079 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:41.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.4 2024-05-08T03:22:41.092 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.133 INFO:teuthology.orchestra.run.smithi177.stdout:60129542189 2024-05-08T03:22:41.138 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:41.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.7 2024-05-08T03:22:41.148 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.3 2024-05-08T03:22:41.151 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.172 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:41.177 INFO:teuthology.orchestra.run.smithi177.stdout:60129542190 2024-05-08T03:22:41.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.6 2024-05-08T03:22:41.187 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.0 2024-05-08T03:22:41.191 DEBUG:teuthology.parallel:result is None 2024-05-08T03:22:41.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T03:22:41.192 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:22:41.514 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:41.515 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:22:41.527 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-08T03:22:40.207181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20476,"ondisk_log_size":20476,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429244,"kb_used_data":19464,"kb_used_omap":19,"kb_used_meta":409644,"kb_avail":754545476,"statfs":{"total":773094113280,"available":772654567424,"internally_reserved":0,"allocated":19931136,"data_stored":10366722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":419475469},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.271267"},"pg_stats":[{"pgid":"3.1f","version":"176'390","reported_seq":3553,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497878+0000","last_change":"2024-05-08T03:22:26.808600+0000","last_active":"2024-05-08T03:22:38.497878+0000","last_peered":"2024-05-08T03:22:38.497878+0000","last_clean":"2024-05-08T03:22:38.497878+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:38.497878+0000","last_undegraded":"2024-05-08T03:22:38.497878+0000","last_fullsized":"2024-05-08T03:22:38.497878+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:52.572436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'356","reported_seq":3749,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509475+0000","last_change":"2024-05-08T03:22:26.803642+0000","last_active":"2024-05-08T03:22:38.509475+0000","last_peered":"2024-05-08T03:22:38.509475+0000","last_clean":"2024-05-08T03:22:38.509475+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:38.509475+0000","last_undegraded":"2024-05-08T03:22:38.509475+0000","last_fullsized":"2024-05-08T03:22:38.509475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:40.060968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"173'542","reported_seq":7327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972369+0000","last_change":"2024-05-08T03:22:23.810462+0000","last_active":"2024-05-08T03:22:37.972369+0000","last_peered":"2024-05-08T03:22:37.972369+0000","last_clean":"2024-05-08T03:22:37.972369+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:37.972369+0000","last_undegraded":"2024-05-08T03:22:37.972369+0000","last_fullsized":"2024-05-08T03:22:37.972369+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:04:19.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1045","reported_seq":4590,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972506+0000","last_change":"2024-05-08T03:22:23.810627+0000","last_active":"2024-05-08T03:22:37.972506+0000","last_peered":"2024-05-08T03:22:37.972506+0000","last_clean":"2024-05-08T03:22:37.972506+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:37.972506+0000","last_undegraded":"2024-05-08T03:22:37.972506+0000","last_fullsized":"2024-05-08T03:22:37.972506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:20:20.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'614","reported_seq":5215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497815+0000","last_change":"2024-05-08T03:22:27.807218+0000","last_active":"2024-05-08T03:22:38.497815+0000","last_peered":"2024-05-08T03:22:38.497815+0000","last_clean":"2024-05-08T03:22:38.497815+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:38.497815+0000","last_undegraded":"2024-05-08T03:22:38.497815+0000","last_fullsized":"2024-05-08T03:22:38.497815+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:55.976815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"174'627","reported_seq":6210,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196110+0000","last_change":"2024-05-08T03:22:16.868895+0000","last_active":"2024-05-08T03:22:38.196110+0000","last_peered":"2024-05-08T03:22:38.196110+0000","last_clean":"2024-05-08T03:22:38.196110+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:38.196110+0000","last_undegraded":"2024-05-08T03:22:38.196110+0000","last_fullsized":"2024-05-08T03:22:38.196110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:27.727588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'1161","reported_seq":4923,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509386+0000","last_change":"2024-05-08T03:22:26.805640+0000","last_active":"2024-05-08T03:22:38.509386+0000","last_peered":"2024-05-08T03:22:38.509386+0000","last_clean":"2024-05-08T03:22:38.509386+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:38.509386+0000","last_undegraded":"2024-05-08T03:22:38.509386+0000","last_fullsized":"2024-05-08T03:22:38.509386+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:19.563789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"176'632","reported_seq":10219,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509464+0000","last_change":"2024-05-08T03:22:27.804667+0000","last_active":"2024-05-08T03:22:38.509464+0000","last_peered":"2024-05-08T03:22:38.509464+0000","last_clean":"2024-05-08T03:22:38.509464+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:38.509464+0000","last_undegraded":"2024-05-08T03:22:38.509464+0000","last_fullsized":"2024-05-08T03:22:38.509464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:01:55.564018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'422","reported_seq":5424,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497794+0000","last_change":"2024-05-08T03:22:24.798989+0000","last_active":"2024-05-08T03:22:38.497794+0000","last_peered":"2024-05-08T03:22:38.497794+0000","last_clean":"2024-05-08T03:22:38.497794+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:38.497794+0000","last_undegraded":"2024-05-08T03:22:38.497794+0000","last_fullsized":"2024-05-08T03:22:38.497794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:33.191227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'296","reported_seq":3949,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972408+0000","last_change":"2024-05-08T03:22:23.808981+0000","last_active":"2024-05-08T03:22:37.972408+0000","last_peered":"2024-05-08T03:22:37.972408+0000","last_clean":"2024-05-08T03:22:37.972408+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:37.972408+0000","last_undegraded":"2024-05-08T03:22:37.972408+0000","last_fullsized":"2024-05-08T03:22:37.972408+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:57:01.587880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"175'435","reported_seq":8316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955970+0000","last_change":"2024-05-08T03:22:27.804313+0000","last_active":"2024-05-08T03:22:37.955970+0000","last_peered":"2024-05-08T03:22:37.955970+0000","last_clean":"2024-05-08T03:22:37.955970+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:37.955970+0000","last_undegraded":"2024-05-08T03:22:37.955970+0000","last_fullsized":"2024-05-08T03:22:37.955970+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:05:37.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"173'905","reported_seq":5644,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196236+0000","last_change":"2024-05-08T03:22:32.932926+0000","last_active":"2024-05-08T03:22:38.196236+0000","last_peered":"2024-05-08T03:22:38.196236+0000","last_clean":"2024-05-08T03:22:38.196236+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:38.196236+0000","last_undegraded":"2024-05-08T03:22:38.196236+0000","last_fullsized":"2024-05-08T03:22:38.196236+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:43.327096+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196019+0000","last_change":"2024-05-08T03:22:36.072861+0000","last_active":"2024-05-08T03:22:38.196019+0000","last_peered":"2024-05-08T03:22:38.196019+0000","last_clean":"2024-05-08T03:22:38.196019+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:38.196019+0000","last_undegraded":"2024-05-08T03:22:38.196019+0000","last_fullsized":"2024-05-08T03:22:38.196019+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:42.430931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'455","reported_seq":4787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972484+0000","last_change":"2024-05-08T03:22:25.810549+0000","last_active":"2024-05-08T03:22:37.972484+0000","last_peered":"2024-05-08T03:22:37.972484+0000","last_clean":"2024-05-08T03:22:37.972484+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:37.972484+0000","last_undegraded":"2024-05-08T03:22:37.972484+0000","last_fullsized":"2024-05-08T03:22:37.972484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:03.796868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196031+0000","last_change":"2024-05-08T03:22:35.952124+0000","last_active":"2024-05-08T03:22:38.196031+0000","last_peered":"2024-05-08T03:22:38.196031+0000","last_clean":"2024-05-08T03:22:38.196031+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:38.196031+0000","last_undegraded":"2024-05-08T03:22:38.196031+0000","last_fullsized":"2024-05-08T03:22:38.196031+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:44.557317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"173'859","reported_seq":5901,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954147+0000","last_change":"2024-05-08T03:22:28.803724+0000","last_active":"2024-05-08T03:22:37.954147+0000","last_peered":"2024-05-08T03:22:37.954147+0000","last_clean":"2024-05-08T03:22:37.954147+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:22:37.954147+0000","last_undegraded":"2024-05-08T03:22:37.954147+0000","last_fullsized":"2024-05-08T03:22:37.954147+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:05.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":233,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196190+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:22:38.196190+0000","last_peered":"2024-05-08T03:22:38.196190+0000","last_clean":"2024-05-08T03:22:38.196190+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:38.196190+0000","last_undegraded":"2024-05-08T03:22:38.196190+0000","last_fullsized":"2024-05-08T03:22:38.196190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"174'332","reported_seq":5754,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509546+0000","last_change":"2024-05-08T03:22:23.806536+0000","last_active":"2024-05-08T03:22:38.509546+0000","last_peered":"2024-05-08T03:22:38.509546+0000","last_clean":"2024-05-08T03:22:38.509546+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:38.509546+0000","last_undegraded":"2024-05-08T03:22:38.509546+0000","last_fullsized":"2024-05-08T03:22:38.509546+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:15.483337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509331+0000","last_change":"2024-05-08T03:22:21.828489+0000","last_active":"2024-05-08T03:22:38.509331+0000","last_peered":"2024-05-08T03:22:38.509331+0000","last_clean":"2024-05-08T03:22:38.509331+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:38.509331+0000","last_undegraded":"2024-05-08T03:22:38.509331+0000","last_fullsized":"2024-05-08T03:22:38.509331+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:12.509619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'1037","reported_seq":5278,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497875+0000","last_change":"2024-05-08T03:22:27.804984+0000","last_active":"2024-05-08T03:22:38.497875+0000","last_peered":"2024-05-08T03:22:38.497875+0000","last_clean":"2024-05-08T03:22:38.497875+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:38.497875+0000","last_undegraded":"2024-05-08T03:22:38.497875+0000","last_fullsized":"2024-05-08T03:22:38.497875+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.103422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'358","reported_seq":4522,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497773+0000","last_change":"2024-05-08T03:22:24.798635+0000","last_active":"2024-05-08T03:22:38.497773+0000","last_peered":"2024-05-08T03:22:38.497773+0000","last_clean":"2024-05-08T03:22:38.497773+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:38.497773+0000","last_undegraded":"2024-05-08T03:22:38.497773+0000","last_fullsized":"2024-05-08T03:22:38.497773+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:03:30.473342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":547,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955741+0000","last_change":"2024-05-08T03:22:28.803643+0000","last_active":"2024-05-08T03:22:37.955741+0000","last_peered":"2024-05-08T03:22:37.955741+0000","last_clean":"2024-05-08T03:22:37.955741+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:37.955741+0000","last_undegraded":"2024-05-08T03:22:37.955741+0000","last_fullsized":"2024-05-08T03:22:37.955741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:46.908323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":369,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954027+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:22:37.954027+0000","last_peered":"2024-05-08T03:22:37.954027+0000","last_clean":"2024-05-08T03:22:37.954027+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:22:37.954027+0000","last_undegraded":"2024-05-08T03:22:37.954027+0000","last_fullsized":"2024-05-08T03:22:37.954027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"174'520","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954105+0000","last_change":"2024-05-08T03:22:16.867571+0000","last_active":"2024-05-08T03:22:37.954105+0000","last_peered":"2024-05-08T03:22:37.954105+0000","last_clean":"2024-05-08T03:22:37.954105+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:22:37.954105+0000","last_undegraded":"2024-05-08T03:22:37.954105+0000","last_fullsized":"2024-05-08T03:22:37.954105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:59.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955724+0000","last_change":"2024-05-08T03:22:22.841205+0000","last_active":"2024-05-08T03:22:37.955724+0000","last_peered":"2024-05-08T03:22:37.955724+0000","last_clean":"2024-05-08T03:22:37.955724+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:37.955724+0000","last_undegraded":"2024-05-08T03:22:37.955724+0000","last_fullsized":"2024-05-08T03:22:37.955724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:22.701410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'1057","reported_seq":9572,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196182+0000","last_change":"2024-05-08T03:22:32.932833+0000","last_active":"2024-05-08T03:22:38.196182+0000","last_peered":"2024-05-08T03:22:38.196182+0000","last_clean":"2024-05-08T03:22:38.196182+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:38.196182+0000","last_undegraded":"2024-05-08T03:22:38.196182+0000","last_fullsized":"2024-05-08T03:22:38.196182+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:28.460551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972255+0000","last_change":"2024-05-08T03:22:22.839738+0000","last_active":"2024-05-08T03:22:37.972255+0000","last_peered":"2024-05-08T03:22:37.972255+0000","last_clean":"2024-05-08T03:22:37.972255+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:37.972255+0000","last_undegraded":"2024-05-08T03:22:37.972255+0000","last_fullsized":"2024-05-08T03:22:37.972255+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:59.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'922","reported_seq":10134,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509397+0000","last_change":"2024-05-08T03:22:26.806179+0000","last_active":"2024-05-08T03:22:38.509397+0000","last_peered":"2024-05-08T03:22:38.509397+0000","last_clean":"2024-05-08T03:22:38.509397+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:38.509397+0000","last_undegraded":"2024-05-08T03:22:38.509397+0000","last_fullsized":"2024-05-08T03:22:38.509397+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:54:33.798889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955748+0000","last_change":"2024-05-08T03:22:22.843526+0000","last_active":"2024-05-08T03:22:37.955748+0000","last_peered":"2024-05-08T03:22:37.955748+0000","last_clean":"2024-05-08T03:22:37.955748+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:37.955748+0000","last_undegraded":"2024-05-08T03:22:37.955748+0000","last_fullsized":"2024-05-08T03:22:37.955748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:14.999728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'543","reported_seq":5933,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509372+0000","last_change":"2024-05-08T03:22:28.807411+0000","last_active":"2024-05-08T03:22:38.509372+0000","last_peered":"2024-05-08T03:22:38.509372+0000","last_clean":"2024-05-08T03:22:38.509372+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:38.509372+0000","last_undegraded":"2024-05-08T03:22:38.509372+0000","last_fullsized":"2024-05-08T03:22:38.509372+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.424295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'544","reported_seq":4058,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195784+0000","last_change":"2024-05-08T03:22:16.871542+0000","last_active":"2024-05-08T03:22:38.195784+0000","last_peered":"2024-05-08T03:22:38.195784+0000","last_clean":"2024-05-08T03:22:38.195784+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:38.195784+0000","last_undegraded":"2024-05-08T03:22:38.195784+0000","last_fullsized":"2024-05-08T03:22:38.195784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:30.948114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"174'620","reported_seq":2412,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195885+0000","last_change":"2024-05-08T03:22:35.953685+0000","last_active":"2024-05-08T03:22:38.195885+0000","last_peered":"2024-05-08T03:22:38.195885+0000","last_clean":"2024-05-08T03:22:38.195885+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:38.195885+0000","last_undegraded":"2024-05-08T03:22:38.195885+0000","last_fullsized":"2024-05-08T03:22:38.195885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:13.225297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'479","reported_seq":7787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509415+0000","last_change":"2024-05-08T03:22:25.810877+0000","last_active":"2024-05-08T03:22:38.509415+0000","last_peered":"2024-05-08T03:22:38.509415+0000","last_clean":"2024-05-08T03:22:38.509415+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:38.509415+0000","last_undegraded":"2024-05-08T03:22:38.509415+0000","last_fullsized":"2024-05-08T03:22:38.509415+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:30.061717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"173'864","reported_seq":5489,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972304+0000","last_change":"2024-05-08T03:22:25.811641+0000","last_active":"2024-05-08T03:22:37.972304+0000","last_peered":"2024-05-08T03:22:37.972304+0000","last_clean":"2024-05-08T03:22:37.972304+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:37.972304+0000","last_undegraded":"2024-05-08T03:22:37.972304+0000","last_fullsized":"2024-05-08T03:22:37.972304+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:29.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'1092","reported_seq":6035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955985+0000","last_change":"2024-05-08T03:22:24.800600+0000","last_active":"2024-05-08T03:22:37.955985+0000","last_peered":"2024-05-08T03:22:37.955985+0000","last_clean":"2024-05-08T03:22:37.955985+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:37.955985+0000","last_undegraded":"2024-05-08T03:22:37.955985+0000","last_fullsized":"2024-05-08T03:22:37.955985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:33.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'681","reported_seq":4829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955856+0000","last_change":"2024-05-08T03:22:16.876674+0000","last_active":"2024-05-08T03:22:37.955856+0000","last_peered":"2024-05-08T03:22:37.955856+0000","last_clean":"2024-05-08T03:22:37.955856+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:37.955856+0000","last_undegraded":"2024-05-08T03:22:37.955856+0000","last_fullsized":"2024-05-08T03:22:37.955856+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:32.370610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"164'612","reported_seq":4242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955904+0000","last_change":"2024-05-08T03:22:16.876490+0000","last_active":"2024-05-08T03:22:37.955904+0000","last_peered":"2024-05-08T03:22:37.955904+0000","last_clean":"2024-05-08T03:22:37.955904+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:37.955904+0000","last_undegraded":"2024-05-08T03:22:37.955904+0000","last_fullsized":"2024-05-08T03:22:37.955904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:15.091431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'522","reported_seq":5146,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195845+0000","last_change":"2024-05-08T03:22:35.953206+0000","last_active":"2024-05-08T03:22:38.195845+0000","last_peered":"2024-05-08T03:22:38.195845+0000","last_clean":"2024-05-08T03:22:38.195845+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:38.195845+0000","last_undegraded":"2024-05-08T03:22:38.195845+0000","last_fullsized":"2024-05-08T03:22:38.195845+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:54.549900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'563","reported_seq":6395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955844+0000","last_change":"2024-05-08T03:22:16.874528+0000","last_active":"2024-05-08T03:22:37.955844+0000","last_peered":"2024-05-08T03:22:37.955844+0000","last_clean":"2024-05-08T03:22:37.955844+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:37.955844+0000","last_undegraded":"2024-05-08T03:22:37.955844+0000","last_fullsized":"2024-05-08T03:22:37.955844+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:22.149690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'430","reported_seq":3492,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497733+0000","last_change":"2024-05-08T03:22:26.807535+0000","last_active":"2024-05-08T03:22:38.497733+0000","last_peered":"2024-05-08T03:22:38.497733+0000","last_clean":"2024-05-08T03:22:38.497733+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:38.497733+0000","last_undegraded":"2024-05-08T03:22:38.497733+0000","last_fullsized":"2024-05-08T03:22:38.497733+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:56.675263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'444","reported_seq":5637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955764+0000","last_change":"2024-05-08T03:22:16.874743+0000","last_active":"2024-05-08T03:22:37.955764+0000","last_peered":"2024-05-08T03:22:37.955764+0000","last_clean":"2024-05-08T03:22:37.955764+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:37.955764+0000","last_undegraded":"2024-05-08T03:22:37.955764+0000","last_fullsized":"2024-05-08T03:22:37.955764+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:40.732000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20359,"ondisk_log_size":20359,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34060,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337780,"statfs":{"total":96636764160,"available":96601886720,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73728,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298112,"statfs":{"total":96636764160,"available":96561266688,"internally_reserved":0,"allocated":2338816,"data_stored":1146783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533398,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38316,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333524,"statfs":{"total":96636764160,"available":96597528576,"internally_reserved":0,"allocated":2326528,"data_stored":1140941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51516,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320324,"statfs":{"total":96636764160,"available":96584011776,"internally_reserved":0,"allocated":2342912,"data_stored":1146802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47360,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324480,"statfs":{"total":96636764160,"available":96588267520,"internally_reserved":0,"allocated":2338816,"data_stored":1146783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82636,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289204,"statfs":{"total":96636764160,"available":96552144896,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41784,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330056,"statfs":{"total":96636764160,"available":96593977344,"internally_reserved":0,"allocated":2338816,"data_stored":1146783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:22:41.529 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:22:41.815 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:41.815 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:22:41.827 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-08T03:22:40.207181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20476,"ondisk_log_size":20476,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429244,"kb_used_data":19464,"kb_used_omap":19,"kb_used_meta":409644,"kb_avail":754545476,"statfs":{"total":773094113280,"available":772654567424,"internally_reserved":0,"allocated":19931136,"data_stored":10366722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":419475469},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.271267"},"pg_stats":[{"pgid":"3.1f","version":"176'390","reported_seq":3553,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497878+0000","last_change":"2024-05-08T03:22:26.808600+0000","last_active":"2024-05-08T03:22:38.497878+0000","last_peered":"2024-05-08T03:22:38.497878+0000","last_clean":"2024-05-08T03:22:38.497878+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:38.497878+0000","last_undegraded":"2024-05-08T03:22:38.497878+0000","last_fullsized":"2024-05-08T03:22:38.497878+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:52.572436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'356","reported_seq":3749,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509475+0000","last_change":"2024-05-08T03:22:26.803642+0000","last_active":"2024-05-08T03:22:38.509475+0000","last_peered":"2024-05-08T03:22:38.509475+0000","last_clean":"2024-05-08T03:22:38.509475+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:38.509475+0000","last_undegraded":"2024-05-08T03:22:38.509475+0000","last_fullsized":"2024-05-08T03:22:38.509475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:40.060968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"173'542","reported_seq":7327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972369+0000","last_change":"2024-05-08T03:22:23.810462+0000","last_active":"2024-05-08T03:22:37.972369+0000","last_peered":"2024-05-08T03:22:37.972369+0000","last_clean":"2024-05-08T03:22:37.972369+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:37.972369+0000","last_undegraded":"2024-05-08T03:22:37.972369+0000","last_fullsized":"2024-05-08T03:22:37.972369+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:04:19.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1045","reported_seq":4590,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972506+0000","last_change":"2024-05-08T03:22:23.810627+0000","last_active":"2024-05-08T03:22:37.972506+0000","last_peered":"2024-05-08T03:22:37.972506+0000","last_clean":"2024-05-08T03:22:37.972506+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:37.972506+0000","last_undegraded":"2024-05-08T03:22:37.972506+0000","last_fullsized":"2024-05-08T03:22:37.972506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:20:20.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'614","reported_seq":5215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497815+0000","last_change":"2024-05-08T03:22:27.807218+0000","last_active":"2024-05-08T03:22:38.497815+0000","last_peered":"2024-05-08T03:22:38.497815+0000","last_clean":"2024-05-08T03:22:38.497815+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:38.497815+0000","last_undegraded":"2024-05-08T03:22:38.497815+0000","last_fullsized":"2024-05-08T03:22:38.497815+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:55.976815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"174'627","reported_seq":6210,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196110+0000","last_change":"2024-05-08T03:22:16.868895+0000","last_active":"2024-05-08T03:22:38.196110+0000","last_peered":"2024-05-08T03:22:38.196110+0000","last_clean":"2024-05-08T03:22:38.196110+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:38.196110+0000","last_undegraded":"2024-05-08T03:22:38.196110+0000","last_fullsized":"2024-05-08T03:22:38.196110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:27.727588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'1161","reported_seq":4923,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509386+0000","last_change":"2024-05-08T03:22:26.805640+0000","last_active":"2024-05-08T03:22:38.509386+0000","last_peered":"2024-05-08T03:22:38.509386+0000","last_clean":"2024-05-08T03:22:38.509386+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:38.509386+0000","last_undegraded":"2024-05-08T03:22:38.509386+0000","last_fullsized":"2024-05-08T03:22:38.509386+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:19.563789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"176'632","reported_seq":10219,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509464+0000","last_change":"2024-05-08T03:22:27.804667+0000","last_active":"2024-05-08T03:22:38.509464+0000","last_peered":"2024-05-08T03:22:38.509464+0000","last_clean":"2024-05-08T03:22:38.509464+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:38.509464+0000","last_undegraded":"2024-05-08T03:22:38.509464+0000","last_fullsized":"2024-05-08T03:22:38.509464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:01:55.564018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'422","reported_seq":5424,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497794+0000","last_change":"2024-05-08T03:22:24.798989+0000","last_active":"2024-05-08T03:22:38.497794+0000","last_peered":"2024-05-08T03:22:38.497794+0000","last_clean":"2024-05-08T03:22:38.497794+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:38.497794+0000","last_undegraded":"2024-05-08T03:22:38.497794+0000","last_fullsized":"2024-05-08T03:22:38.497794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:33.191227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'296","reported_seq":3949,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972408+0000","last_change":"2024-05-08T03:22:23.808981+0000","last_active":"2024-05-08T03:22:37.972408+0000","last_peered":"2024-05-08T03:22:37.972408+0000","last_clean":"2024-05-08T03:22:37.972408+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:37.972408+0000","last_undegraded":"2024-05-08T03:22:37.972408+0000","last_fullsized":"2024-05-08T03:22:37.972408+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:57:01.587880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"175'435","reported_seq":8316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955970+0000","last_change":"2024-05-08T03:22:27.804313+0000","last_active":"2024-05-08T03:22:37.955970+0000","last_peered":"2024-05-08T03:22:37.955970+0000","last_clean":"2024-05-08T03:22:37.955970+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:37.955970+0000","last_undegraded":"2024-05-08T03:22:37.955970+0000","last_fullsized":"2024-05-08T03:22:37.955970+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:05:37.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"173'905","reported_seq":5644,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196236+0000","last_change":"2024-05-08T03:22:32.932926+0000","last_active":"2024-05-08T03:22:38.196236+0000","last_peered":"2024-05-08T03:22:38.196236+0000","last_clean":"2024-05-08T03:22:38.196236+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:38.196236+0000","last_undegraded":"2024-05-08T03:22:38.196236+0000","last_fullsized":"2024-05-08T03:22:38.196236+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:43.327096+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196019+0000","last_change":"2024-05-08T03:22:36.072861+0000","last_active":"2024-05-08T03:22:38.196019+0000","last_peered":"2024-05-08T03:22:38.196019+0000","last_clean":"2024-05-08T03:22:38.196019+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:38.196019+0000","last_undegraded":"2024-05-08T03:22:38.196019+0000","last_fullsized":"2024-05-08T03:22:38.196019+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:42.430931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'455","reported_seq":4787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972484+0000","last_change":"2024-05-08T03:22:25.810549+0000","last_active":"2024-05-08T03:22:37.972484+0000","last_peered":"2024-05-08T03:22:37.972484+0000","last_clean":"2024-05-08T03:22:37.972484+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:37.972484+0000","last_undegraded":"2024-05-08T03:22:37.972484+0000","last_fullsized":"2024-05-08T03:22:37.972484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:03.796868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196031+0000","last_change":"2024-05-08T03:22:35.952124+0000","last_active":"2024-05-08T03:22:38.196031+0000","last_peered":"2024-05-08T03:22:38.196031+0000","last_clean":"2024-05-08T03:22:38.196031+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:38.196031+0000","last_undegraded":"2024-05-08T03:22:38.196031+0000","last_fullsized":"2024-05-08T03:22:38.196031+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:44.557317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"173'859","reported_seq":5901,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954147+0000","last_change":"2024-05-08T03:22:28.803724+0000","last_active":"2024-05-08T03:22:37.954147+0000","last_peered":"2024-05-08T03:22:37.954147+0000","last_clean":"2024-05-08T03:22:37.954147+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:22:37.954147+0000","last_undegraded":"2024-05-08T03:22:37.954147+0000","last_fullsized":"2024-05-08T03:22:37.954147+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:05.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":233,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196190+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:22:38.196190+0000","last_peered":"2024-05-08T03:22:38.196190+0000","last_clean":"2024-05-08T03:22:38.196190+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:38.196190+0000","last_undegraded":"2024-05-08T03:22:38.196190+0000","last_fullsized":"2024-05-08T03:22:38.196190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"174'332","reported_seq":5754,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509546+0000","last_change":"2024-05-08T03:22:23.806536+0000","last_active":"2024-05-08T03:22:38.509546+0000","last_peered":"2024-05-08T03:22:38.509546+0000","last_clean":"2024-05-08T03:22:38.509546+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:38.509546+0000","last_undegraded":"2024-05-08T03:22:38.509546+0000","last_fullsized":"2024-05-08T03:22:38.509546+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:15.483337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509331+0000","last_change":"2024-05-08T03:22:21.828489+0000","last_active":"2024-05-08T03:22:38.509331+0000","last_peered":"2024-05-08T03:22:38.509331+0000","last_clean":"2024-05-08T03:22:38.509331+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:38.509331+0000","last_undegraded":"2024-05-08T03:22:38.509331+0000","last_fullsized":"2024-05-08T03:22:38.509331+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:12.509619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'1037","reported_seq":5278,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497875+0000","last_change":"2024-05-08T03:22:27.804984+0000","last_active":"2024-05-08T03:22:38.497875+0000","last_peered":"2024-05-08T03:22:38.497875+0000","last_clean":"2024-05-08T03:22:38.497875+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:38.497875+0000","last_undegraded":"2024-05-08T03:22:38.497875+0000","last_fullsized":"2024-05-08T03:22:38.497875+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.103422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'358","reported_seq":4522,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497773+0000","last_change":"2024-05-08T03:22:24.798635+0000","last_active":"2024-05-08T03:22:38.497773+0000","last_peered":"2024-05-08T03:22:38.497773+0000","last_clean":"2024-05-08T03:22:38.497773+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:38.497773+0000","last_undegraded":"2024-05-08T03:22:38.497773+0000","last_fullsized":"2024-05-08T03:22:38.497773+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:03:30.473342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":547,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955741+0000","last_change":"2024-05-08T03:22:28.803643+0000","last_active":"2024-05-08T03:22:37.955741+0000","last_peered":"2024-05-08T03:22:37.955741+0000","last_clean":"2024-05-08T03:22:37.955741+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:37.955741+0000","last_undegraded":"2024-05-08T03:22:37.955741+0000","last_fullsized":"2024-05-08T03:22:37.955741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:46.908323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":369,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954027+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:22:37.954027+0000","last_peered":"2024-05-08T03:22:37.954027+0000","last_clean":"2024-05-08T03:22:37.954027+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:22:37.954027+0000","last_undegraded":"2024-05-08T03:22:37.954027+0000","last_fullsized":"2024-05-08T03:22:37.954027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"174'520","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954105+0000","last_change":"2024-05-08T03:22:16.867571+0000","last_active":"2024-05-08T03:22:37.954105+0000","last_peered":"2024-05-08T03:22:37.954105+0000","last_clean":"2024-05-08T03:22:37.954105+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:22:37.954105+0000","last_undegraded":"2024-05-08T03:22:37.954105+0000","last_fullsized":"2024-05-08T03:22:37.954105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:59.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955724+0000","last_change":"2024-05-08T03:22:22.841205+0000","last_active":"2024-05-08T03:22:37.955724+0000","last_peered":"2024-05-08T03:22:37.955724+0000","last_clean":"2024-05-08T03:22:37.955724+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:37.955724+0000","last_undegraded":"2024-05-08T03:22:37.955724+0000","last_fullsized":"2024-05-08T03:22:37.955724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:22.701410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'1057","reported_seq":9572,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196182+0000","last_change":"2024-05-08T03:22:32.932833+0000","last_active":"2024-05-08T03:22:38.196182+0000","last_peered":"2024-05-08T03:22:38.196182+0000","last_clean":"2024-05-08T03:22:38.196182+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:38.196182+0000","last_undegraded":"2024-05-08T03:22:38.196182+0000","last_fullsized":"2024-05-08T03:22:38.196182+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:28.460551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972255+0000","last_change":"2024-05-08T03:22:22.839738+0000","last_active":"2024-05-08T03:22:37.972255+0000","last_peered":"2024-05-08T03:22:37.972255+0000","last_clean":"2024-05-08T03:22:37.972255+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:37.972255+0000","last_undegraded":"2024-05-08T03:22:37.972255+0000","last_fullsized":"2024-05-08T03:22:37.972255+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:59.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'922","reported_seq":10134,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509397+0000","last_change":"2024-05-08T03:22:26.806179+0000","last_active":"2024-05-08T03:22:38.509397+0000","last_peered":"2024-05-08T03:22:38.509397+0000","last_clean":"2024-05-08T03:22:38.509397+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:38.509397+0000","last_undegraded":"2024-05-08T03:22:38.509397+0000","last_fullsized":"2024-05-08T03:22:38.509397+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:54:33.798889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955748+0000","last_change":"2024-05-08T03:22:22.843526+0000","last_active":"2024-05-08T03:22:37.955748+0000","last_peered":"2024-05-08T03:22:37.955748+0000","last_clean":"2024-05-08T03:22:37.955748+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:37.955748+0000","last_undegraded":"2024-05-08T03:22:37.955748+0000","last_fullsized":"2024-05-08T03:22:37.955748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:14.999728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'543","reported_seq":5933,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509372+0000","last_change":"2024-05-08T03:22:28.807411+0000","last_active":"2024-05-08T03:22:38.509372+0000","last_peered":"2024-05-08T03:22:38.509372+0000","last_clean":"2024-05-08T03:22:38.509372+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:38.509372+0000","last_undegraded":"2024-05-08T03:22:38.509372+0000","last_fullsized":"2024-05-08T03:22:38.509372+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.424295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'544","reported_seq":4058,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195784+0000","last_change":"2024-05-08T03:22:16.871542+0000","last_active":"2024-05-08T03:22:38.195784+0000","last_peered":"2024-05-08T03:22:38.195784+0000","last_clean":"2024-05-08T03:22:38.195784+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:38.195784+0000","last_undegraded":"2024-05-08T03:22:38.195784+0000","last_fullsized":"2024-05-08T03:22:38.195784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:30.948114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"174'620","reported_seq":2412,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195885+0000","last_change":"2024-05-08T03:22:35.953685+0000","last_active":"2024-05-08T03:22:38.195885+0000","last_peered":"2024-05-08T03:22:38.195885+0000","last_clean":"2024-05-08T03:22:38.195885+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:38.195885+0000","last_undegraded":"2024-05-08T03:22:38.195885+0000","last_fullsized":"2024-05-08T03:22:38.195885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:13.225297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'479","reported_seq":7787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509415+0000","last_change":"2024-05-08T03:22:25.810877+0000","last_active":"2024-05-08T03:22:38.509415+0000","last_peered":"2024-05-08T03:22:38.509415+0000","last_clean":"2024-05-08T03:22:38.509415+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:38.509415+0000","last_undegraded":"2024-05-08T03:22:38.509415+0000","last_fullsized":"2024-05-08T03:22:38.509415+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:30.061717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"173'864","reported_seq":5489,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972304+0000","last_change":"2024-05-08T03:22:25.811641+0000","last_active":"2024-05-08T03:22:37.972304+0000","last_peered":"2024-05-08T03:22:37.972304+0000","last_clean":"2024-05-08T03:22:37.972304+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:37.972304+0000","last_undegraded":"2024-05-08T03:22:37.972304+0000","last_fullsized":"2024-05-08T03:22:37.972304+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:29.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'1092","reported_seq":6035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955985+0000","last_change":"2024-05-08T03:22:24.800600+0000","last_active":"2024-05-08T03:22:37.955985+0000","last_peered":"2024-05-08T03:22:37.955985+0000","last_clean":"2024-05-08T03:22:37.955985+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:37.955985+0000","last_undegraded":"2024-05-08T03:22:37.955985+0000","last_fullsized":"2024-05-08T03:22:37.955985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:33.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'681","reported_seq":4829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955856+0000","last_change":"2024-05-08T03:22:16.876674+0000","last_active":"2024-05-08T03:22:37.955856+0000","last_peered":"2024-05-08T03:22:37.955856+0000","last_clean":"2024-05-08T03:22:37.955856+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:37.955856+0000","last_undegraded":"2024-05-08T03:22:37.955856+0000","last_fullsized":"2024-05-08T03:22:37.955856+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:32.370610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"164'612","reported_seq":4242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955904+0000","last_change":"2024-05-08T03:22:16.876490+0000","last_active":"2024-05-08T03:22:37.955904+0000","last_peered":"2024-05-08T03:22:37.955904+0000","last_clean":"2024-05-08T03:22:37.955904+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:37.955904+0000","last_undegraded":"2024-05-08T03:22:37.955904+0000","last_fullsized":"2024-05-08T03:22:37.955904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:15.091431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'522","reported_seq":5146,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195845+0000","last_change":"2024-05-08T03:22:35.953206+0000","last_active":"2024-05-08T03:22:38.195845+0000","last_peered":"2024-05-08T03:22:38.195845+0000","last_clean":"2024-05-08T03:22:38.195845+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:38.195845+0000","last_undegraded":"2024-05-08T03:22:38.195845+0000","last_fullsized":"2024-05-08T03:22:38.195845+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:54.549900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'563","reported_seq":6395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955844+0000","last_change":"2024-05-08T03:22:16.874528+0000","last_active":"2024-05-08T03:22:37.955844+0000","last_peered":"2024-05-08T03:22:37.955844+0000","last_clean":"2024-05-08T03:22:37.955844+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:37.955844+0000","last_undegraded":"2024-05-08T03:22:37.955844+0000","last_fullsized":"2024-05-08T03:22:37.955844+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:22.149690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'430","reported_seq":3492,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497733+0000","last_change":"2024-05-08T03:22:26.807535+0000","last_active":"2024-05-08T03:22:38.497733+0000","last_peered":"2024-05-08T03:22:38.497733+0000","last_clean":"2024-05-08T03:22:38.497733+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:38.497733+0000","last_undegraded":"2024-05-08T03:22:38.497733+0000","last_fullsized":"2024-05-08T03:22:38.497733+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:56.675263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'444","reported_seq":5637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955764+0000","last_change":"2024-05-08T03:22:16.874743+0000","last_active":"2024-05-08T03:22:37.955764+0000","last_peered":"2024-05-08T03:22:37.955764+0000","last_clean":"2024-05-08T03:22:37.955764+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:37.955764+0000","last_undegraded":"2024-05-08T03:22:37.955764+0000","last_fullsized":"2024-05-08T03:22:37.955764+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:40.732000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20359,"ondisk_log_size":20359,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34060,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337780,"statfs":{"total":96636764160,"available":96601886720,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73728,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298112,"statfs":{"total":96636764160,"available":96561266688,"internally_reserved":0,"allocated":2338816,"data_stored":1146783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533398,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38316,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333524,"statfs":{"total":96636764160,"available":96597528576,"internally_reserved":0,"allocated":2326528,"data_stored":1140941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51516,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320324,"statfs":{"total":96636764160,"available":96584011776,"internally_reserved":0,"allocated":2342912,"data_stored":1146802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47360,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324480,"statfs":{"total":96636764160,"available":96588267520,"internally_reserved":0,"allocated":2338816,"data_stored":1146783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82636,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289204,"statfs":{"total":96636764160,"available":96552144896,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41784,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330056,"statfs":{"total":96636764160,"available":96593977344,"internally_reserved":0,"allocated":2338816,"data_stored":1146783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:22:41.829 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T03:22:41.829 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-05-08T03:22:42.139 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T03:22:42.149 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T03:22:42.150 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:22:42.433 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:42.434 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:22:42.445 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-08T03:22:42.207681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20476,"ondisk_log_size":20476,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429284,"kb_used_data":19512,"kb_used_omap":19,"kb_used_meta":409644,"kb_avail":754545436,"statfs":{"total":773094113280,"available":772654526464,"internally_reserved":0,"allocated":19980288,"data_stored":10390090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":419475469},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.262342"},"pg_stats":[{"pgid":"3.1f","version":"176'390","reported_seq":3553,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497878+0000","last_change":"2024-05-08T03:22:26.808600+0000","last_active":"2024-05-08T03:22:38.497878+0000","last_peered":"2024-05-08T03:22:38.497878+0000","last_clean":"2024-05-08T03:22:38.497878+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:38.497878+0000","last_undegraded":"2024-05-08T03:22:38.497878+0000","last_fullsized":"2024-05-08T03:22:38.497878+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:52.572436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'356","reported_seq":3749,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509475+0000","last_change":"2024-05-08T03:22:26.803642+0000","last_active":"2024-05-08T03:22:38.509475+0000","last_peered":"2024-05-08T03:22:38.509475+0000","last_clean":"2024-05-08T03:22:38.509475+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:38.509475+0000","last_undegraded":"2024-05-08T03:22:38.509475+0000","last_fullsized":"2024-05-08T03:22:38.509475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:40.060968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"173'542","reported_seq":7327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972369+0000","last_change":"2024-05-08T03:22:23.810462+0000","last_active":"2024-05-08T03:22:37.972369+0000","last_peered":"2024-05-08T03:22:37.972369+0000","last_clean":"2024-05-08T03:22:37.972369+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:37.972369+0000","last_undegraded":"2024-05-08T03:22:37.972369+0000","last_fullsized":"2024-05-08T03:22:37.972369+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:04:19.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1045","reported_seq":4590,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972506+0000","last_change":"2024-05-08T03:22:23.810627+0000","last_active":"2024-05-08T03:22:37.972506+0000","last_peered":"2024-05-08T03:22:37.972506+0000","last_clean":"2024-05-08T03:22:37.972506+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:37.972506+0000","last_undegraded":"2024-05-08T03:22:37.972506+0000","last_fullsized":"2024-05-08T03:22:37.972506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:20:20.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'614","reported_seq":5215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497815+0000","last_change":"2024-05-08T03:22:27.807218+0000","last_active":"2024-05-08T03:22:38.497815+0000","last_peered":"2024-05-08T03:22:38.497815+0000","last_clean":"2024-05-08T03:22:38.497815+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:38.497815+0000","last_undegraded":"2024-05-08T03:22:38.497815+0000","last_fullsized":"2024-05-08T03:22:38.497815+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:55.976815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"174'627","reported_seq":6212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882710+0000","last_change":"2024-05-08T03:22:16.868895+0000","last_active":"2024-05-08T03:22:38.882710+0000","last_peered":"2024-05-08T03:22:38.882710+0000","last_clean":"2024-05-08T03:22:38.882710+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:38.882710+0000","last_undegraded":"2024-05-08T03:22:38.882710+0000","last_fullsized":"2024-05-08T03:22:38.882710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:27.727588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'1161","reported_seq":4923,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509386+0000","last_change":"2024-05-08T03:22:26.805640+0000","last_active":"2024-05-08T03:22:38.509386+0000","last_peered":"2024-05-08T03:22:38.509386+0000","last_clean":"2024-05-08T03:22:38.509386+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:38.509386+0000","last_undegraded":"2024-05-08T03:22:38.509386+0000","last_fullsized":"2024-05-08T03:22:38.509386+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:19.563789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"176'632","reported_seq":10219,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509464+0000","last_change":"2024-05-08T03:22:27.804667+0000","last_active":"2024-05-08T03:22:38.509464+0000","last_peered":"2024-05-08T03:22:38.509464+0000","last_clean":"2024-05-08T03:22:38.509464+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:38.509464+0000","last_undegraded":"2024-05-08T03:22:38.509464+0000","last_fullsized":"2024-05-08T03:22:38.509464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:01:55.564018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'422","reported_seq":5424,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497794+0000","last_change":"2024-05-08T03:22:24.798989+0000","last_active":"2024-05-08T03:22:38.497794+0000","last_peered":"2024-05-08T03:22:38.497794+0000","last_clean":"2024-05-08T03:22:38.497794+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:38.497794+0000","last_undegraded":"2024-05-08T03:22:38.497794+0000","last_fullsized":"2024-05-08T03:22:38.497794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:33.191227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'296","reported_seq":3949,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972408+0000","last_change":"2024-05-08T03:22:23.808981+0000","last_active":"2024-05-08T03:22:37.972408+0000","last_peered":"2024-05-08T03:22:37.972408+0000","last_clean":"2024-05-08T03:22:37.972408+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:37.972408+0000","last_undegraded":"2024-05-08T03:22:37.972408+0000","last_fullsized":"2024-05-08T03:22:37.972408+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:57:01.587880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"175'435","reported_seq":8316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955970+0000","last_change":"2024-05-08T03:22:27.804313+0000","last_active":"2024-05-08T03:22:37.955970+0000","last_peered":"2024-05-08T03:22:37.955970+0000","last_clean":"2024-05-08T03:22:37.955970+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:37.955970+0000","last_undegraded":"2024-05-08T03:22:37.955970+0000","last_fullsized":"2024-05-08T03:22:37.955970+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:05:37.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"173'905","reported_seq":5646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882960+0000","last_change":"2024-05-08T03:22:32.932926+0000","last_active":"2024-05-08T03:22:38.882960+0000","last_peered":"2024-05-08T03:22:38.882960+0000","last_clean":"2024-05-08T03:22:38.882960+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:38.882960+0000","last_undegraded":"2024-05-08T03:22:38.882960+0000","last_fullsized":"2024-05-08T03:22:38.882960+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:43.327096+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196019+0000","last_change":"2024-05-08T03:22:36.072861+0000","last_active":"2024-05-08T03:22:38.196019+0000","last_peered":"2024-05-08T03:22:38.196019+0000","last_clean":"2024-05-08T03:22:38.196019+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:38.196019+0000","last_undegraded":"2024-05-08T03:22:38.196019+0000","last_fullsized":"2024-05-08T03:22:38.196019+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:42.430931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'455","reported_seq":4787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972484+0000","last_change":"2024-05-08T03:22:25.810549+0000","last_active":"2024-05-08T03:22:37.972484+0000","last_peered":"2024-05-08T03:22:37.972484+0000","last_clean":"2024-05-08T03:22:37.972484+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:37.972484+0000","last_undegraded":"2024-05-08T03:22:37.972484+0000","last_fullsized":"2024-05-08T03:22:37.972484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:03.796868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196031+0000","last_change":"2024-05-08T03:22:35.952124+0000","last_active":"2024-05-08T03:22:38.196031+0000","last_peered":"2024-05-08T03:22:38.196031+0000","last_clean":"2024-05-08T03:22:38.196031+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:38.196031+0000","last_undegraded":"2024-05-08T03:22:38.196031+0000","last_fullsized":"2024-05-08T03:22:38.196031+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:44.557317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"173'859","reported_seq":5901,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954147+0000","last_change":"2024-05-08T03:22:28.803724+0000","last_active":"2024-05-08T03:22:37.954147+0000","last_peered":"2024-05-08T03:22:37.954147+0000","last_clean":"2024-05-08T03:22:37.954147+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:22:37.954147+0000","last_undegraded":"2024-05-08T03:22:37.954147+0000","last_fullsized":"2024-05-08T03:22:37.954147+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:05.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":236,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882909+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:22:38.882909+0000","last_peered":"2024-05-08T03:22:38.882909+0000","last_clean":"2024-05-08T03:22:38.882909+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:38.882909+0000","last_undegraded":"2024-05-08T03:22:38.882909+0000","last_fullsized":"2024-05-08T03:22:38.882909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"174'332","reported_seq":5754,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509546+0000","last_change":"2024-05-08T03:22:23.806536+0000","last_active":"2024-05-08T03:22:38.509546+0000","last_peered":"2024-05-08T03:22:38.509546+0000","last_clean":"2024-05-08T03:22:38.509546+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:38.509546+0000","last_undegraded":"2024-05-08T03:22:38.509546+0000","last_fullsized":"2024-05-08T03:22:38.509546+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:15.483337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509331+0000","last_change":"2024-05-08T03:22:21.828489+0000","last_active":"2024-05-08T03:22:38.509331+0000","last_peered":"2024-05-08T03:22:38.509331+0000","last_clean":"2024-05-08T03:22:38.509331+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:38.509331+0000","last_undegraded":"2024-05-08T03:22:38.509331+0000","last_fullsized":"2024-05-08T03:22:38.509331+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:12.509619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'1037","reported_seq":5278,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497875+0000","last_change":"2024-05-08T03:22:27.804984+0000","last_active":"2024-05-08T03:22:38.497875+0000","last_peered":"2024-05-08T03:22:38.497875+0000","last_clean":"2024-05-08T03:22:38.497875+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:38.497875+0000","last_undegraded":"2024-05-08T03:22:38.497875+0000","last_fullsized":"2024-05-08T03:22:38.497875+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.103422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'358","reported_seq":4522,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497773+0000","last_change":"2024-05-08T03:22:24.798635+0000","last_active":"2024-05-08T03:22:38.497773+0000","last_peered":"2024-05-08T03:22:38.497773+0000","last_clean":"2024-05-08T03:22:38.497773+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:38.497773+0000","last_undegraded":"2024-05-08T03:22:38.497773+0000","last_fullsized":"2024-05-08T03:22:38.497773+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:03:30.473342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":547,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955741+0000","last_change":"2024-05-08T03:22:28.803643+0000","last_active":"2024-05-08T03:22:37.955741+0000","last_peered":"2024-05-08T03:22:37.955741+0000","last_clean":"2024-05-08T03:22:37.955741+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:37.955741+0000","last_undegraded":"2024-05-08T03:22:37.955741+0000","last_fullsized":"2024-05-08T03:22:37.955741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:46.908323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":369,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954027+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:22:37.954027+0000","last_peered":"2024-05-08T03:22:37.954027+0000","last_clean":"2024-05-08T03:22:37.954027+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:22:37.954027+0000","last_undegraded":"2024-05-08T03:22:37.954027+0000","last_fullsized":"2024-05-08T03:22:37.954027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"174'520","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954105+0000","last_change":"2024-05-08T03:22:16.867571+0000","last_active":"2024-05-08T03:22:37.954105+0000","last_peered":"2024-05-08T03:22:37.954105+0000","last_clean":"2024-05-08T03:22:37.954105+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:22:37.954105+0000","last_undegraded":"2024-05-08T03:22:37.954105+0000","last_fullsized":"2024-05-08T03:22:37.954105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:59.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955724+0000","last_change":"2024-05-08T03:22:22.841205+0000","last_active":"2024-05-08T03:22:37.955724+0000","last_peered":"2024-05-08T03:22:37.955724+0000","last_clean":"2024-05-08T03:22:37.955724+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:37.955724+0000","last_undegraded":"2024-05-08T03:22:37.955724+0000","last_fullsized":"2024-05-08T03:22:37.955724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:22.701410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'1057","reported_seq":9574,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882764+0000","last_change":"2024-05-08T03:22:32.932833+0000","last_active":"2024-05-08T03:22:38.882764+0000","last_peered":"2024-05-08T03:22:38.882764+0000","last_clean":"2024-05-08T03:22:38.882764+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:38.882764+0000","last_undegraded":"2024-05-08T03:22:38.882764+0000","last_fullsized":"2024-05-08T03:22:38.882764+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:28.460551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972255+0000","last_change":"2024-05-08T03:22:22.839738+0000","last_active":"2024-05-08T03:22:37.972255+0000","last_peered":"2024-05-08T03:22:37.972255+0000","last_clean":"2024-05-08T03:22:37.972255+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:37.972255+0000","last_undegraded":"2024-05-08T03:22:37.972255+0000","last_fullsized":"2024-05-08T03:22:37.972255+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:59.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'922","reported_seq":10134,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509397+0000","last_change":"2024-05-08T03:22:26.806179+0000","last_active":"2024-05-08T03:22:38.509397+0000","last_peered":"2024-05-08T03:22:38.509397+0000","last_clean":"2024-05-08T03:22:38.509397+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:38.509397+0000","last_undegraded":"2024-05-08T03:22:38.509397+0000","last_fullsized":"2024-05-08T03:22:38.509397+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:54:33.798889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955748+0000","last_change":"2024-05-08T03:22:22.843526+0000","last_active":"2024-05-08T03:22:37.955748+0000","last_peered":"2024-05-08T03:22:37.955748+0000","last_clean":"2024-05-08T03:22:37.955748+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:37.955748+0000","last_undegraded":"2024-05-08T03:22:37.955748+0000","last_fullsized":"2024-05-08T03:22:37.955748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:14.999728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'543","reported_seq":5933,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509372+0000","last_change":"2024-05-08T03:22:28.807411+0000","last_active":"2024-05-08T03:22:38.509372+0000","last_peered":"2024-05-08T03:22:38.509372+0000","last_clean":"2024-05-08T03:22:38.509372+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:38.509372+0000","last_undegraded":"2024-05-08T03:22:38.509372+0000","last_fullsized":"2024-05-08T03:22:38.509372+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.424295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'544","reported_seq":4060,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882767+0000","last_change":"2024-05-08T03:22:16.871542+0000","last_active":"2024-05-08T03:22:38.882767+0000","last_peered":"2024-05-08T03:22:38.882767+0000","last_clean":"2024-05-08T03:22:38.882767+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:38.882767+0000","last_undegraded":"2024-05-08T03:22:38.882767+0000","last_fullsized":"2024-05-08T03:22:38.882767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:30.948114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"174'620","reported_seq":2412,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195885+0000","last_change":"2024-05-08T03:22:35.953685+0000","last_active":"2024-05-08T03:22:38.195885+0000","last_peered":"2024-05-08T03:22:38.195885+0000","last_clean":"2024-05-08T03:22:38.195885+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:38.195885+0000","last_undegraded":"2024-05-08T03:22:38.195885+0000","last_fullsized":"2024-05-08T03:22:38.195885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:13.225297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'479","reported_seq":7787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509415+0000","last_change":"2024-05-08T03:22:25.810877+0000","last_active":"2024-05-08T03:22:38.509415+0000","last_peered":"2024-05-08T03:22:38.509415+0000","last_clean":"2024-05-08T03:22:38.509415+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:38.509415+0000","last_undegraded":"2024-05-08T03:22:38.509415+0000","last_fullsized":"2024-05-08T03:22:38.509415+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:30.061717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"173'864","reported_seq":5489,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972304+0000","last_change":"2024-05-08T03:22:25.811641+0000","last_active":"2024-05-08T03:22:37.972304+0000","last_peered":"2024-05-08T03:22:37.972304+0000","last_clean":"2024-05-08T03:22:37.972304+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:37.972304+0000","last_undegraded":"2024-05-08T03:22:37.972304+0000","last_fullsized":"2024-05-08T03:22:37.972304+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:29.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'1092","reported_seq":6035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955985+0000","last_change":"2024-05-08T03:22:24.800600+0000","last_active":"2024-05-08T03:22:37.955985+0000","last_peered":"2024-05-08T03:22:37.955985+0000","last_clean":"2024-05-08T03:22:37.955985+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:37.955985+0000","last_undegraded":"2024-05-08T03:22:37.955985+0000","last_fullsized":"2024-05-08T03:22:37.955985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:33.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'681","reported_seq":4829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955856+0000","last_change":"2024-05-08T03:22:16.876674+0000","last_active":"2024-05-08T03:22:37.955856+0000","last_peered":"2024-05-08T03:22:37.955856+0000","last_clean":"2024-05-08T03:22:37.955856+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:37.955856+0000","last_undegraded":"2024-05-08T03:22:37.955856+0000","last_fullsized":"2024-05-08T03:22:37.955856+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:32.370610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"164'612","reported_seq":4242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955904+0000","last_change":"2024-05-08T03:22:16.876490+0000","last_active":"2024-05-08T03:22:37.955904+0000","last_peered":"2024-05-08T03:22:37.955904+0000","last_clean":"2024-05-08T03:22:37.955904+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:37.955904+0000","last_undegraded":"2024-05-08T03:22:37.955904+0000","last_fullsized":"2024-05-08T03:22:37.955904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:15.091431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'522","reported_seq":5146,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195845+0000","last_change":"2024-05-08T03:22:35.953206+0000","last_active":"2024-05-08T03:22:38.195845+0000","last_peered":"2024-05-08T03:22:38.195845+0000","last_clean":"2024-05-08T03:22:38.195845+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:38.195845+0000","last_undegraded":"2024-05-08T03:22:38.195845+0000","last_fullsized":"2024-05-08T03:22:38.195845+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:54.549900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'563","reported_seq":6395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955844+0000","last_change":"2024-05-08T03:22:16.874528+0000","last_active":"2024-05-08T03:22:37.955844+0000","last_peered":"2024-05-08T03:22:37.955844+0000","last_clean":"2024-05-08T03:22:37.955844+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:37.955844+0000","last_undegraded":"2024-05-08T03:22:37.955844+0000","last_fullsized":"2024-05-08T03:22:37.955844+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:22.149690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'430","reported_seq":3492,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497733+0000","last_change":"2024-05-08T03:22:26.807535+0000","last_active":"2024-05-08T03:22:38.497733+0000","last_peered":"2024-05-08T03:22:38.497733+0000","last_clean":"2024-05-08T03:22:38.497733+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:38.497733+0000","last_undegraded":"2024-05-08T03:22:38.497733+0000","last_fullsized":"2024-05-08T03:22:38.497733+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:56.675263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'444","reported_seq":5637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955764+0000","last_change":"2024-05-08T03:22:16.874743+0000","last_active":"2024-05-08T03:22:37.955764+0000","last_peered":"2024-05-08T03:22:37.955764+0000","last_clean":"2024-05-08T03:22:37.955764+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:37.955764+0000","last_undegraded":"2024-05-08T03:22:37.955764+0000","last_fullsized":"2024-05-08T03:22:37.955764+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:40.732000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20359,"ondisk_log_size":20359,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34060,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337780,"statfs":{"total":96636764160,"available":96601886720,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533398,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38316,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333524,"statfs":{"total":96636764160,"available":96597528576,"internally_reserved":0,"allocated":2326528,"data_stored":1140941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2355200,"data_stored":1152644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82636,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289204,"statfs":{"total":96636764160,"available":96552144896,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41800,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330040,"statfs":{"total":96636764160,"available":96593960960,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:22:42.447 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:22:42.732 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:42.734 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:22:42.746 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-08T03:22:42.207681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20476,"ondisk_log_size":20476,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429284,"kb_used_data":19512,"kb_used_omap":19,"kb_used_meta":409644,"kb_avail":754545436,"statfs":{"total":773094113280,"available":772654526464,"internally_reserved":0,"allocated":19980288,"data_stored":10390090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":419475469},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.262342"},"pg_stats":[{"pgid":"3.1f","version":"176'390","reported_seq":3553,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497878+0000","last_change":"2024-05-08T03:22:26.808600+0000","last_active":"2024-05-08T03:22:38.497878+0000","last_peered":"2024-05-08T03:22:38.497878+0000","last_clean":"2024-05-08T03:22:38.497878+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:38.497878+0000","last_undegraded":"2024-05-08T03:22:38.497878+0000","last_fullsized":"2024-05-08T03:22:38.497878+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:52.572436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'356","reported_seq":3749,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509475+0000","last_change":"2024-05-08T03:22:26.803642+0000","last_active":"2024-05-08T03:22:38.509475+0000","last_peered":"2024-05-08T03:22:38.509475+0000","last_clean":"2024-05-08T03:22:38.509475+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:38.509475+0000","last_undegraded":"2024-05-08T03:22:38.509475+0000","last_fullsized":"2024-05-08T03:22:38.509475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:40.060968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"173'542","reported_seq":7327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972369+0000","last_change":"2024-05-08T03:22:23.810462+0000","last_active":"2024-05-08T03:22:37.972369+0000","last_peered":"2024-05-08T03:22:37.972369+0000","last_clean":"2024-05-08T03:22:37.972369+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:37.972369+0000","last_undegraded":"2024-05-08T03:22:37.972369+0000","last_fullsized":"2024-05-08T03:22:37.972369+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:04:19.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1045","reported_seq":4590,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972506+0000","last_change":"2024-05-08T03:22:23.810627+0000","last_active":"2024-05-08T03:22:37.972506+0000","last_peered":"2024-05-08T03:22:37.972506+0000","last_clean":"2024-05-08T03:22:37.972506+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:37.972506+0000","last_undegraded":"2024-05-08T03:22:37.972506+0000","last_fullsized":"2024-05-08T03:22:37.972506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:20:20.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'614","reported_seq":5215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497815+0000","last_change":"2024-05-08T03:22:27.807218+0000","last_active":"2024-05-08T03:22:38.497815+0000","last_peered":"2024-05-08T03:22:38.497815+0000","last_clean":"2024-05-08T03:22:38.497815+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:38.497815+0000","last_undegraded":"2024-05-08T03:22:38.497815+0000","last_fullsized":"2024-05-08T03:22:38.497815+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:55.976815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"174'627","reported_seq":6212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882710+0000","last_change":"2024-05-08T03:22:16.868895+0000","last_active":"2024-05-08T03:22:38.882710+0000","last_peered":"2024-05-08T03:22:38.882710+0000","last_clean":"2024-05-08T03:22:38.882710+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:38.882710+0000","last_undegraded":"2024-05-08T03:22:38.882710+0000","last_fullsized":"2024-05-08T03:22:38.882710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:27.727588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'1161","reported_seq":4923,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509386+0000","last_change":"2024-05-08T03:22:26.805640+0000","last_active":"2024-05-08T03:22:38.509386+0000","last_peered":"2024-05-08T03:22:38.509386+0000","last_clean":"2024-05-08T03:22:38.509386+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:38.509386+0000","last_undegraded":"2024-05-08T03:22:38.509386+0000","last_fullsized":"2024-05-08T03:22:38.509386+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:19.563789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"176'632","reported_seq":10219,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509464+0000","last_change":"2024-05-08T03:22:27.804667+0000","last_active":"2024-05-08T03:22:38.509464+0000","last_peered":"2024-05-08T03:22:38.509464+0000","last_clean":"2024-05-08T03:22:38.509464+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:38.509464+0000","last_undegraded":"2024-05-08T03:22:38.509464+0000","last_fullsized":"2024-05-08T03:22:38.509464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:01:55.564018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'422","reported_seq":5424,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497794+0000","last_change":"2024-05-08T03:22:24.798989+0000","last_active":"2024-05-08T03:22:38.497794+0000","last_peered":"2024-05-08T03:22:38.497794+0000","last_clean":"2024-05-08T03:22:38.497794+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:38.497794+0000","last_undegraded":"2024-05-08T03:22:38.497794+0000","last_fullsized":"2024-05-08T03:22:38.497794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:33.191227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'296","reported_seq":3949,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972408+0000","last_change":"2024-05-08T03:22:23.808981+0000","last_active":"2024-05-08T03:22:37.972408+0000","last_peered":"2024-05-08T03:22:37.972408+0000","last_clean":"2024-05-08T03:22:37.972408+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:37.972408+0000","last_undegraded":"2024-05-08T03:22:37.972408+0000","last_fullsized":"2024-05-08T03:22:37.972408+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:57:01.587880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"175'435","reported_seq":8316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955970+0000","last_change":"2024-05-08T03:22:27.804313+0000","last_active":"2024-05-08T03:22:37.955970+0000","last_peered":"2024-05-08T03:22:37.955970+0000","last_clean":"2024-05-08T03:22:37.955970+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:37.955970+0000","last_undegraded":"2024-05-08T03:22:37.955970+0000","last_fullsized":"2024-05-08T03:22:37.955970+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:05:37.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"173'905","reported_seq":5646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882960+0000","last_change":"2024-05-08T03:22:32.932926+0000","last_active":"2024-05-08T03:22:38.882960+0000","last_peered":"2024-05-08T03:22:38.882960+0000","last_clean":"2024-05-08T03:22:38.882960+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:38.882960+0000","last_undegraded":"2024-05-08T03:22:38.882960+0000","last_fullsized":"2024-05-08T03:22:38.882960+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:43.327096+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196019+0000","last_change":"2024-05-08T03:22:36.072861+0000","last_active":"2024-05-08T03:22:38.196019+0000","last_peered":"2024-05-08T03:22:38.196019+0000","last_clean":"2024-05-08T03:22:38.196019+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:38.196019+0000","last_undegraded":"2024-05-08T03:22:38.196019+0000","last_fullsized":"2024-05-08T03:22:38.196019+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:42.430931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'455","reported_seq":4787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972484+0000","last_change":"2024-05-08T03:22:25.810549+0000","last_active":"2024-05-08T03:22:37.972484+0000","last_peered":"2024-05-08T03:22:37.972484+0000","last_clean":"2024-05-08T03:22:37.972484+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:37.972484+0000","last_undegraded":"2024-05-08T03:22:37.972484+0000","last_fullsized":"2024-05-08T03:22:37.972484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:03.796868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196031+0000","last_change":"2024-05-08T03:22:35.952124+0000","last_active":"2024-05-08T03:22:38.196031+0000","last_peered":"2024-05-08T03:22:38.196031+0000","last_clean":"2024-05-08T03:22:38.196031+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:38.196031+0000","last_undegraded":"2024-05-08T03:22:38.196031+0000","last_fullsized":"2024-05-08T03:22:38.196031+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:44.557317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"173'859","reported_seq":5901,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954147+0000","last_change":"2024-05-08T03:22:28.803724+0000","last_active":"2024-05-08T03:22:37.954147+0000","last_peered":"2024-05-08T03:22:37.954147+0000","last_clean":"2024-05-08T03:22:37.954147+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:22:37.954147+0000","last_undegraded":"2024-05-08T03:22:37.954147+0000","last_fullsized":"2024-05-08T03:22:37.954147+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:05.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":236,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882909+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:22:38.882909+0000","last_peered":"2024-05-08T03:22:38.882909+0000","last_clean":"2024-05-08T03:22:38.882909+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:38.882909+0000","last_undegraded":"2024-05-08T03:22:38.882909+0000","last_fullsized":"2024-05-08T03:22:38.882909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"174'332","reported_seq":5754,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509546+0000","last_change":"2024-05-08T03:22:23.806536+0000","last_active":"2024-05-08T03:22:38.509546+0000","last_peered":"2024-05-08T03:22:38.509546+0000","last_clean":"2024-05-08T03:22:38.509546+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:38.509546+0000","last_undegraded":"2024-05-08T03:22:38.509546+0000","last_fullsized":"2024-05-08T03:22:38.509546+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:15.483337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509331+0000","last_change":"2024-05-08T03:22:21.828489+0000","last_active":"2024-05-08T03:22:38.509331+0000","last_peered":"2024-05-08T03:22:38.509331+0000","last_clean":"2024-05-08T03:22:38.509331+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:38.509331+0000","last_undegraded":"2024-05-08T03:22:38.509331+0000","last_fullsized":"2024-05-08T03:22:38.509331+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:12.509619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'1037","reported_seq":5278,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497875+0000","last_change":"2024-05-08T03:22:27.804984+0000","last_active":"2024-05-08T03:22:38.497875+0000","last_peered":"2024-05-08T03:22:38.497875+0000","last_clean":"2024-05-08T03:22:38.497875+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:38.497875+0000","last_undegraded":"2024-05-08T03:22:38.497875+0000","last_fullsized":"2024-05-08T03:22:38.497875+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.103422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'358","reported_seq":4522,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497773+0000","last_change":"2024-05-08T03:22:24.798635+0000","last_active":"2024-05-08T03:22:38.497773+0000","last_peered":"2024-05-08T03:22:38.497773+0000","last_clean":"2024-05-08T03:22:38.497773+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:38.497773+0000","last_undegraded":"2024-05-08T03:22:38.497773+0000","last_fullsized":"2024-05-08T03:22:38.497773+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:03:30.473342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":547,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955741+0000","last_change":"2024-05-08T03:22:28.803643+0000","last_active":"2024-05-08T03:22:37.955741+0000","last_peered":"2024-05-08T03:22:37.955741+0000","last_clean":"2024-05-08T03:22:37.955741+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:37.955741+0000","last_undegraded":"2024-05-08T03:22:37.955741+0000","last_fullsized":"2024-05-08T03:22:37.955741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:46.908323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":369,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954027+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:22:37.954027+0000","last_peered":"2024-05-08T03:22:37.954027+0000","last_clean":"2024-05-08T03:22:37.954027+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:22:37.954027+0000","last_undegraded":"2024-05-08T03:22:37.954027+0000","last_fullsized":"2024-05-08T03:22:37.954027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"174'520","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954105+0000","last_change":"2024-05-08T03:22:16.867571+0000","last_active":"2024-05-08T03:22:37.954105+0000","last_peered":"2024-05-08T03:22:37.954105+0000","last_clean":"2024-05-08T03:22:37.954105+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:22:37.954105+0000","last_undegraded":"2024-05-08T03:22:37.954105+0000","last_fullsized":"2024-05-08T03:22:37.954105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:59.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955724+0000","last_change":"2024-05-08T03:22:22.841205+0000","last_active":"2024-05-08T03:22:37.955724+0000","last_peered":"2024-05-08T03:22:37.955724+0000","last_clean":"2024-05-08T03:22:37.955724+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:37.955724+0000","last_undegraded":"2024-05-08T03:22:37.955724+0000","last_fullsized":"2024-05-08T03:22:37.955724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:22.701410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'1057","reported_seq":9574,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882764+0000","last_change":"2024-05-08T03:22:32.932833+0000","last_active":"2024-05-08T03:22:38.882764+0000","last_peered":"2024-05-08T03:22:38.882764+0000","last_clean":"2024-05-08T03:22:38.882764+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:38.882764+0000","last_undegraded":"2024-05-08T03:22:38.882764+0000","last_fullsized":"2024-05-08T03:22:38.882764+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:28.460551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972255+0000","last_change":"2024-05-08T03:22:22.839738+0000","last_active":"2024-05-08T03:22:37.972255+0000","last_peered":"2024-05-08T03:22:37.972255+0000","last_clean":"2024-05-08T03:22:37.972255+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:37.972255+0000","last_undegraded":"2024-05-08T03:22:37.972255+0000","last_fullsized":"2024-05-08T03:22:37.972255+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:59.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'922","reported_seq":10134,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509397+0000","last_change":"2024-05-08T03:22:26.806179+0000","last_active":"2024-05-08T03:22:38.509397+0000","last_peered":"2024-05-08T03:22:38.509397+0000","last_clean":"2024-05-08T03:22:38.509397+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:38.509397+0000","last_undegraded":"2024-05-08T03:22:38.509397+0000","last_fullsized":"2024-05-08T03:22:38.509397+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:54:33.798889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955748+0000","last_change":"2024-05-08T03:22:22.843526+0000","last_active":"2024-05-08T03:22:37.955748+0000","last_peered":"2024-05-08T03:22:37.955748+0000","last_clean":"2024-05-08T03:22:37.955748+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:37.955748+0000","last_undegraded":"2024-05-08T03:22:37.955748+0000","last_fullsized":"2024-05-08T03:22:37.955748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:14.999728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'543","reported_seq":5933,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509372+0000","last_change":"2024-05-08T03:22:28.807411+0000","last_active":"2024-05-08T03:22:38.509372+0000","last_peered":"2024-05-08T03:22:38.509372+0000","last_clean":"2024-05-08T03:22:38.509372+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:38.509372+0000","last_undegraded":"2024-05-08T03:22:38.509372+0000","last_fullsized":"2024-05-08T03:22:38.509372+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.424295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'544","reported_seq":4060,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882767+0000","last_change":"2024-05-08T03:22:16.871542+0000","last_active":"2024-05-08T03:22:38.882767+0000","last_peered":"2024-05-08T03:22:38.882767+0000","last_clean":"2024-05-08T03:22:38.882767+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:38.882767+0000","last_undegraded":"2024-05-08T03:22:38.882767+0000","last_fullsized":"2024-05-08T03:22:38.882767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:30.948114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"174'620","reported_seq":2412,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195885+0000","last_change":"2024-05-08T03:22:35.953685+0000","last_active":"2024-05-08T03:22:38.195885+0000","last_peered":"2024-05-08T03:22:38.195885+0000","last_clean":"2024-05-08T03:22:38.195885+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:38.195885+0000","last_undegraded":"2024-05-08T03:22:38.195885+0000","last_fullsized":"2024-05-08T03:22:38.195885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:13.225297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'479","reported_seq":7787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509415+0000","last_change":"2024-05-08T03:22:25.810877+0000","last_active":"2024-05-08T03:22:38.509415+0000","last_peered":"2024-05-08T03:22:38.509415+0000","last_clean":"2024-05-08T03:22:38.509415+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:38.509415+0000","last_undegraded":"2024-05-08T03:22:38.509415+0000","last_fullsized":"2024-05-08T03:22:38.509415+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:30.061717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"173'864","reported_seq":5489,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972304+0000","last_change":"2024-05-08T03:22:25.811641+0000","last_active":"2024-05-08T03:22:37.972304+0000","last_peered":"2024-05-08T03:22:37.972304+0000","last_clean":"2024-05-08T03:22:37.972304+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:37.972304+0000","last_undegraded":"2024-05-08T03:22:37.972304+0000","last_fullsized":"2024-05-08T03:22:37.972304+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:29.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'1092","reported_seq":6035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955985+0000","last_change":"2024-05-08T03:22:24.800600+0000","last_active":"2024-05-08T03:22:37.955985+0000","last_peered":"2024-05-08T03:22:37.955985+0000","last_clean":"2024-05-08T03:22:37.955985+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:37.955985+0000","last_undegraded":"2024-05-08T03:22:37.955985+0000","last_fullsized":"2024-05-08T03:22:37.955985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:33.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'681","reported_seq":4829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955856+0000","last_change":"2024-05-08T03:22:16.876674+0000","last_active":"2024-05-08T03:22:37.955856+0000","last_peered":"2024-05-08T03:22:37.955856+0000","last_clean":"2024-05-08T03:22:37.955856+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:37.955856+0000","last_undegraded":"2024-05-08T03:22:37.955856+0000","last_fullsized":"2024-05-08T03:22:37.955856+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:32.370610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"164'612","reported_seq":4242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955904+0000","last_change":"2024-05-08T03:22:16.876490+0000","last_active":"2024-05-08T03:22:37.955904+0000","last_peered":"2024-05-08T03:22:37.955904+0000","last_clean":"2024-05-08T03:22:37.955904+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:37.955904+0000","last_undegraded":"2024-05-08T03:22:37.955904+0000","last_fullsized":"2024-05-08T03:22:37.955904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:15.091431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'522","reported_seq":5146,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195845+0000","last_change":"2024-05-08T03:22:35.953206+0000","last_active":"2024-05-08T03:22:38.195845+0000","last_peered":"2024-05-08T03:22:38.195845+0000","last_clean":"2024-05-08T03:22:38.195845+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:38.195845+0000","last_undegraded":"2024-05-08T03:22:38.195845+0000","last_fullsized":"2024-05-08T03:22:38.195845+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:54.549900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'563","reported_seq":6395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955844+0000","last_change":"2024-05-08T03:22:16.874528+0000","last_active":"2024-05-08T03:22:37.955844+0000","last_peered":"2024-05-08T03:22:37.955844+0000","last_clean":"2024-05-08T03:22:37.955844+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:37.955844+0000","last_undegraded":"2024-05-08T03:22:37.955844+0000","last_fullsized":"2024-05-08T03:22:37.955844+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:22.149690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'430","reported_seq":3492,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497733+0000","last_change":"2024-05-08T03:22:26.807535+0000","last_active":"2024-05-08T03:22:38.497733+0000","last_peered":"2024-05-08T03:22:38.497733+0000","last_clean":"2024-05-08T03:22:38.497733+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:38.497733+0000","last_undegraded":"2024-05-08T03:22:38.497733+0000","last_fullsized":"2024-05-08T03:22:38.497733+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:56.675263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'444","reported_seq":5637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955764+0000","last_change":"2024-05-08T03:22:16.874743+0000","last_active":"2024-05-08T03:22:37.955764+0000","last_peered":"2024-05-08T03:22:37.955764+0000","last_clean":"2024-05-08T03:22:37.955764+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:37.955764+0000","last_undegraded":"2024-05-08T03:22:37.955764+0000","last_fullsized":"2024-05-08T03:22:37.955764+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:40.732000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20359,"ondisk_log_size":20359,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34060,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337780,"statfs":{"total":96636764160,"available":96601886720,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533398,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38316,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333524,"statfs":{"total":96636764160,"available":96597528576,"internally_reserved":0,"allocated":2326528,"data_stored":1140941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2355200,"data_stored":1152644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82636,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289204,"statfs":{"total":96636764160,"available":96552144896,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41800,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330040,"statfs":{"total":96636764160,"available":96593960960,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:22:42.748 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T03:22:42.748 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:22:43.036 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:43.036 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:22:43.048 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-08T03:22:42.207681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20476,"ondisk_log_size":20476,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429284,"kb_used_data":19512,"kb_used_omap":19,"kb_used_meta":409644,"kb_avail":754545436,"statfs":{"total":773094113280,"available":772654526464,"internally_reserved":0,"allocated":19980288,"data_stored":10390090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":419475469},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.262342"},"pg_stats":[{"pgid":"3.1f","version":"176'390","reported_seq":3553,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497878+0000","last_change":"2024-05-08T03:22:26.808600+0000","last_active":"2024-05-08T03:22:38.497878+0000","last_peered":"2024-05-08T03:22:38.497878+0000","last_clean":"2024-05-08T03:22:38.497878+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:38.497878+0000","last_undegraded":"2024-05-08T03:22:38.497878+0000","last_fullsized":"2024-05-08T03:22:38.497878+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:52.572436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'356","reported_seq":3749,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509475+0000","last_change":"2024-05-08T03:22:26.803642+0000","last_active":"2024-05-08T03:22:38.509475+0000","last_peered":"2024-05-08T03:22:38.509475+0000","last_clean":"2024-05-08T03:22:38.509475+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:38.509475+0000","last_undegraded":"2024-05-08T03:22:38.509475+0000","last_fullsized":"2024-05-08T03:22:38.509475+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:24:40.060968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"173'542","reported_seq":7327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972369+0000","last_change":"2024-05-08T03:22:23.810462+0000","last_active":"2024-05-08T03:22:37.972369+0000","last_peered":"2024-05-08T03:22:37.972369+0000","last_clean":"2024-05-08T03:22:37.972369+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:37.972369+0000","last_undegraded":"2024-05-08T03:22:37.972369+0000","last_fullsized":"2024-05-08T03:22:37.972369+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:04:19.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1045","reported_seq":4590,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972506+0000","last_change":"2024-05-08T03:22:23.810627+0000","last_active":"2024-05-08T03:22:37.972506+0000","last_peered":"2024-05-08T03:22:37.972506+0000","last_clean":"2024-05-08T03:22:37.972506+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:37.972506+0000","last_undegraded":"2024-05-08T03:22:37.972506+0000","last_fullsized":"2024-05-08T03:22:37.972506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:20:20.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'614","reported_seq":5215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497815+0000","last_change":"2024-05-08T03:22:27.807218+0000","last_active":"2024-05-08T03:22:38.497815+0000","last_peered":"2024-05-08T03:22:38.497815+0000","last_clean":"2024-05-08T03:22:38.497815+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:38.497815+0000","last_undegraded":"2024-05-08T03:22:38.497815+0000","last_fullsized":"2024-05-08T03:22:38.497815+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:55.976815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"174'627","reported_seq":6212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882710+0000","last_change":"2024-05-08T03:22:16.868895+0000","last_active":"2024-05-08T03:22:38.882710+0000","last_peered":"2024-05-08T03:22:38.882710+0000","last_clean":"2024-05-08T03:22:38.882710+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:38.882710+0000","last_undegraded":"2024-05-08T03:22:38.882710+0000","last_fullsized":"2024-05-08T03:22:38.882710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:27.727588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'1161","reported_seq":4923,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509386+0000","last_change":"2024-05-08T03:22:26.805640+0000","last_active":"2024-05-08T03:22:38.509386+0000","last_peered":"2024-05-08T03:22:38.509386+0000","last_clean":"2024-05-08T03:22:38.509386+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:38.509386+0000","last_undegraded":"2024-05-08T03:22:38.509386+0000","last_fullsized":"2024-05-08T03:22:38.509386+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:19.563789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"176'632","reported_seq":10219,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509464+0000","last_change":"2024-05-08T03:22:27.804667+0000","last_active":"2024-05-08T03:22:38.509464+0000","last_peered":"2024-05-08T03:22:38.509464+0000","last_clean":"2024-05-08T03:22:38.509464+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:38.509464+0000","last_undegraded":"2024-05-08T03:22:38.509464+0000","last_fullsized":"2024-05-08T03:22:38.509464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:01:55.564018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'422","reported_seq":5424,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497794+0000","last_change":"2024-05-08T03:22:24.798989+0000","last_active":"2024-05-08T03:22:38.497794+0000","last_peered":"2024-05-08T03:22:38.497794+0000","last_clean":"2024-05-08T03:22:38.497794+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:38.497794+0000","last_undegraded":"2024-05-08T03:22:38.497794+0000","last_fullsized":"2024-05-08T03:22:38.497794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:33.191227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'296","reported_seq":3949,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972408+0000","last_change":"2024-05-08T03:22:23.808981+0000","last_active":"2024-05-08T03:22:37.972408+0000","last_peered":"2024-05-08T03:22:37.972408+0000","last_clean":"2024-05-08T03:22:37.972408+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:37.972408+0000","last_undegraded":"2024-05-08T03:22:37.972408+0000","last_fullsized":"2024-05-08T03:22:37.972408+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:57:01.587880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"175'435","reported_seq":8316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955970+0000","last_change":"2024-05-08T03:22:27.804313+0000","last_active":"2024-05-08T03:22:37.955970+0000","last_peered":"2024-05-08T03:22:37.955970+0000","last_clean":"2024-05-08T03:22:37.955970+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:37.955970+0000","last_undegraded":"2024-05-08T03:22:37.955970+0000","last_fullsized":"2024-05-08T03:22:37.955970+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:05:37.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"173'905","reported_seq":5646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882960+0000","last_change":"2024-05-08T03:22:32.932926+0000","last_active":"2024-05-08T03:22:38.882960+0000","last_peered":"2024-05-08T03:22:38.882960+0000","last_clean":"2024-05-08T03:22:38.882960+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:38.882960+0000","last_undegraded":"2024-05-08T03:22:38.882960+0000","last_fullsized":"2024-05-08T03:22:38.882960+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:43.327096+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196019+0000","last_change":"2024-05-08T03:22:36.072861+0000","last_active":"2024-05-08T03:22:38.196019+0000","last_peered":"2024-05-08T03:22:38.196019+0000","last_clean":"2024-05-08T03:22:38.196019+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:38.196019+0000","last_undegraded":"2024-05-08T03:22:38.196019+0000","last_fullsized":"2024-05-08T03:22:38.196019+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:42.430931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'455","reported_seq":4787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972484+0000","last_change":"2024-05-08T03:22:25.810549+0000","last_active":"2024-05-08T03:22:37.972484+0000","last_peered":"2024-05-08T03:22:37.972484+0000","last_clean":"2024-05-08T03:22:37.972484+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:37.972484+0000","last_undegraded":"2024-05-08T03:22:37.972484+0000","last_fullsized":"2024-05-08T03:22:37.972484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:03.796868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196031+0000","last_change":"2024-05-08T03:22:35.952124+0000","last_active":"2024-05-08T03:22:38.196031+0000","last_peered":"2024-05-08T03:22:38.196031+0000","last_clean":"2024-05-08T03:22:38.196031+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:38.196031+0000","last_undegraded":"2024-05-08T03:22:38.196031+0000","last_fullsized":"2024-05-08T03:22:38.196031+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:44.557317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"173'859","reported_seq":5901,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954147+0000","last_change":"2024-05-08T03:22:28.803724+0000","last_active":"2024-05-08T03:22:37.954147+0000","last_peered":"2024-05-08T03:22:37.954147+0000","last_clean":"2024-05-08T03:22:37.954147+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:22:37.954147+0000","last_undegraded":"2024-05-08T03:22:37.954147+0000","last_fullsized":"2024-05-08T03:22:37.954147+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:05.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":236,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882909+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:22:38.882909+0000","last_peered":"2024-05-08T03:22:38.882909+0000","last_clean":"2024-05-08T03:22:38.882909+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:38.882909+0000","last_undegraded":"2024-05-08T03:22:38.882909+0000","last_fullsized":"2024-05-08T03:22:38.882909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"174'332","reported_seq":5754,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509546+0000","last_change":"2024-05-08T03:22:23.806536+0000","last_active":"2024-05-08T03:22:38.509546+0000","last_peered":"2024-05-08T03:22:38.509546+0000","last_clean":"2024-05-08T03:22:38.509546+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:38.509546+0000","last_undegraded":"2024-05-08T03:22:38.509546+0000","last_fullsized":"2024-05-08T03:22:38.509546+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:15.483337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509331+0000","last_change":"2024-05-08T03:22:21.828489+0000","last_active":"2024-05-08T03:22:38.509331+0000","last_peered":"2024-05-08T03:22:38.509331+0000","last_clean":"2024-05-08T03:22:38.509331+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:38.509331+0000","last_undegraded":"2024-05-08T03:22:38.509331+0000","last_fullsized":"2024-05-08T03:22:38.509331+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:12.509619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'1037","reported_seq":5278,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497875+0000","last_change":"2024-05-08T03:22:27.804984+0000","last_active":"2024-05-08T03:22:38.497875+0000","last_peered":"2024-05-08T03:22:38.497875+0000","last_clean":"2024-05-08T03:22:38.497875+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:38.497875+0000","last_undegraded":"2024-05-08T03:22:38.497875+0000","last_fullsized":"2024-05-08T03:22:38.497875+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.103422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"174'358","reported_seq":4522,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497773+0000","last_change":"2024-05-08T03:22:24.798635+0000","last_active":"2024-05-08T03:22:38.497773+0000","last_peered":"2024-05-08T03:22:38.497773+0000","last_clean":"2024-05-08T03:22:38.497773+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:38.497773+0000","last_undegraded":"2024-05-08T03:22:38.497773+0000","last_fullsized":"2024-05-08T03:22:38.497773+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:03:30.473342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":547,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955741+0000","last_change":"2024-05-08T03:22:28.803643+0000","last_active":"2024-05-08T03:22:37.955741+0000","last_peered":"2024-05-08T03:22:37.955741+0000","last_clean":"2024-05-08T03:22:37.955741+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:37.955741+0000","last_undegraded":"2024-05-08T03:22:37.955741+0000","last_fullsized":"2024-05-08T03:22:37.955741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:46.908323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":369,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954027+0000","last_change":"2024-05-08T03:19:12.731499+0000","last_active":"2024-05-08T03:22:37.954027+0000","last_peered":"2024-05-08T03:22:37.954027+0000","last_clean":"2024-05-08T03:22:37.954027+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:22:37.954027+0000","last_undegraded":"2024-05-08T03:22:37.954027+0000","last_fullsized":"2024-05-08T03:22:37.954027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"174'520","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954105+0000","last_change":"2024-05-08T03:22:16.867571+0000","last_active":"2024-05-08T03:22:37.954105+0000","last_peered":"2024-05-08T03:22:37.954105+0000","last_clean":"2024-05-08T03:22:37.954105+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:22:37.954105+0000","last_undegraded":"2024-05-08T03:22:37.954105+0000","last_fullsized":"2024-05-08T03:22:37.954105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:59.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955724+0000","last_change":"2024-05-08T03:22:22.841205+0000","last_active":"2024-05-08T03:22:37.955724+0000","last_peered":"2024-05-08T03:22:37.955724+0000","last_clean":"2024-05-08T03:22:37.955724+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:37.955724+0000","last_undegraded":"2024-05-08T03:22:37.955724+0000","last_fullsized":"2024-05-08T03:22:37.955724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:22.701410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'1057","reported_seq":9574,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882764+0000","last_change":"2024-05-08T03:22:32.932833+0000","last_active":"2024-05-08T03:22:38.882764+0000","last_peered":"2024-05-08T03:22:38.882764+0000","last_clean":"2024-05-08T03:22:38.882764+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:38.882764+0000","last_undegraded":"2024-05-08T03:22:38.882764+0000","last_fullsized":"2024-05-08T03:22:38.882764+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:28.460551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972255+0000","last_change":"2024-05-08T03:22:22.839738+0000","last_active":"2024-05-08T03:22:37.972255+0000","last_peered":"2024-05-08T03:22:37.972255+0000","last_clean":"2024-05-08T03:22:37.972255+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:37.972255+0000","last_undegraded":"2024-05-08T03:22:37.972255+0000","last_fullsized":"2024-05-08T03:22:37.972255+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:59.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'922","reported_seq":10134,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509397+0000","last_change":"2024-05-08T03:22:26.806179+0000","last_active":"2024-05-08T03:22:38.509397+0000","last_peered":"2024-05-08T03:22:38.509397+0000","last_clean":"2024-05-08T03:22:38.509397+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:38.509397+0000","last_undegraded":"2024-05-08T03:22:38.509397+0000","last_fullsized":"2024-05-08T03:22:38.509397+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:54:33.798889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955748+0000","last_change":"2024-05-08T03:22:22.843526+0000","last_active":"2024-05-08T03:22:37.955748+0000","last_peered":"2024-05-08T03:22:37.955748+0000","last_clean":"2024-05-08T03:22:37.955748+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:37.955748+0000","last_undegraded":"2024-05-08T03:22:37.955748+0000","last_fullsized":"2024-05-08T03:22:37.955748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:14.999728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'543","reported_seq":5933,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509372+0000","last_change":"2024-05-08T03:22:28.807411+0000","last_active":"2024-05-08T03:22:38.509372+0000","last_peered":"2024-05-08T03:22:38.509372+0000","last_clean":"2024-05-08T03:22:38.509372+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:38.509372+0000","last_undegraded":"2024-05-08T03:22:38.509372+0000","last_fullsized":"2024-05-08T03:22:38.509372+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.424295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'544","reported_seq":4060,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882767+0000","last_change":"2024-05-08T03:22:16.871542+0000","last_active":"2024-05-08T03:22:38.882767+0000","last_peered":"2024-05-08T03:22:38.882767+0000","last_clean":"2024-05-08T03:22:38.882767+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:38.882767+0000","last_undegraded":"2024-05-08T03:22:38.882767+0000","last_fullsized":"2024-05-08T03:22:38.882767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:30.948114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"174'620","reported_seq":2412,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195885+0000","last_change":"2024-05-08T03:22:35.953685+0000","last_active":"2024-05-08T03:22:38.195885+0000","last_peered":"2024-05-08T03:22:38.195885+0000","last_clean":"2024-05-08T03:22:38.195885+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:38.195885+0000","last_undegraded":"2024-05-08T03:22:38.195885+0000","last_fullsized":"2024-05-08T03:22:38.195885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:13.225297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'479","reported_seq":7787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509415+0000","last_change":"2024-05-08T03:22:25.810877+0000","last_active":"2024-05-08T03:22:38.509415+0000","last_peered":"2024-05-08T03:22:38.509415+0000","last_clean":"2024-05-08T03:22:38.509415+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:38.509415+0000","last_undegraded":"2024-05-08T03:22:38.509415+0000","last_fullsized":"2024-05-08T03:22:38.509415+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:30.061717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"173'864","reported_seq":5489,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972304+0000","last_change":"2024-05-08T03:22:25.811641+0000","last_active":"2024-05-08T03:22:37.972304+0000","last_peered":"2024-05-08T03:22:37.972304+0000","last_clean":"2024-05-08T03:22:37.972304+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:37.972304+0000","last_undegraded":"2024-05-08T03:22:37.972304+0000","last_fullsized":"2024-05-08T03:22:37.972304+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:29.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'1092","reported_seq":6035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955985+0000","last_change":"2024-05-08T03:22:24.800600+0000","last_active":"2024-05-08T03:22:37.955985+0000","last_peered":"2024-05-08T03:22:37.955985+0000","last_clean":"2024-05-08T03:22:37.955985+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:37.955985+0000","last_undegraded":"2024-05-08T03:22:37.955985+0000","last_fullsized":"2024-05-08T03:22:37.955985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:33.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'681","reported_seq":4829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955856+0000","last_change":"2024-05-08T03:22:16.876674+0000","last_active":"2024-05-08T03:22:37.955856+0000","last_peered":"2024-05-08T03:22:37.955856+0000","last_clean":"2024-05-08T03:22:37.955856+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:37.955856+0000","last_undegraded":"2024-05-08T03:22:37.955856+0000","last_fullsized":"2024-05-08T03:22:37.955856+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:32.370610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"164'612","reported_seq":4242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955904+0000","last_change":"2024-05-08T03:22:16.876490+0000","last_active":"2024-05-08T03:22:37.955904+0000","last_peered":"2024-05-08T03:22:37.955904+0000","last_clean":"2024-05-08T03:22:37.955904+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:37.955904+0000","last_undegraded":"2024-05-08T03:22:37.955904+0000","last_fullsized":"2024-05-08T03:22:37.955904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:15.091431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'522","reported_seq":5146,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195845+0000","last_change":"2024-05-08T03:22:35.953206+0000","last_active":"2024-05-08T03:22:38.195845+0000","last_peered":"2024-05-08T03:22:38.195845+0000","last_clean":"2024-05-08T03:22:38.195845+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:38.195845+0000","last_undegraded":"2024-05-08T03:22:38.195845+0000","last_fullsized":"2024-05-08T03:22:38.195845+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:54.549900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'563","reported_seq":6395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955844+0000","last_change":"2024-05-08T03:22:16.874528+0000","last_active":"2024-05-08T03:22:37.955844+0000","last_peered":"2024-05-08T03:22:37.955844+0000","last_clean":"2024-05-08T03:22:37.955844+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:37.955844+0000","last_undegraded":"2024-05-08T03:22:37.955844+0000","last_fullsized":"2024-05-08T03:22:37.955844+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:22.149690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'430","reported_seq":3492,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497733+0000","last_change":"2024-05-08T03:22:26.807535+0000","last_active":"2024-05-08T03:22:38.497733+0000","last_peered":"2024-05-08T03:22:38.497733+0000","last_clean":"2024-05-08T03:22:38.497733+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:38.497733+0000","last_undegraded":"2024-05-08T03:22:38.497733+0000","last_fullsized":"2024-05-08T03:22:38.497733+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:56.675263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'444","reported_seq":5637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955764+0000","last_change":"2024-05-08T03:22:16.874743+0000","last_active":"2024-05-08T03:22:37.955764+0000","last_peered":"2024-05-08T03:22:37.955764+0000","last_clean":"2024-05-08T03:22:37.955764+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:37.955764+0000","last_undegraded":"2024-05-08T03:22:37.955764+0000","last_fullsized":"2024-05-08T03:22:37.955764+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:40.732000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20359,"ondisk_log_size":20359,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34060,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337780,"statfs":{"total":96636764160,"available":96601886720,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533398,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38316,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333524,"statfs":{"total":96636764160,"available":96597528576,"internally_reserved":0,"allocated":2326528,"data_stored":1140941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2355200,"data_stored":1152644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82636,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289204,"statfs":{"total":96636764160,"available":96552144896,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41800,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330040,"statfs":{"total":96636764160,"available":96593960960,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:22:43.050 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T03:22:43.339 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:43.339 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":195,"fsid":"a7e973ac-16c0-41c5-817e-e3e8f10aad7b","created":"2024-05-08T03:18:59.968487+0000","modified":"2024-05-08T03:22:37.950144+0000","last_up_change":"2024-05-08T03:21:03.827528+0000","last_in_change":"2024-05-08T03:22:21.787702+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T03:19:11.648360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T03:19:12.206654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T03:19:19.077332+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":49,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5bdb4f77-60d7-46da-a930-5450259e00b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6809","nonce":2599410006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6811","nonce":2599410006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6815","nonce":2599410006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2599410006},{"type":"v1","addr":"172.21.15.177:6813","nonce":2599410006}]},"public_addr":"172.21.15.177:6809/2599410006","cluster_addr":"172.21.15.177:6811/2599410006","heartbeat_back_addr":"172.21.15.177:6815/2599410006","heartbeat_front_addr":"172.21.15.177:6813/2599410006","state":["exists","up"]},{"osd":1,"uuid":"1d56bac7-6aad-4b3f-a0b5-8116bbeaeeb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6823","nonce":3471641477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6827","nonce":3471641477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6831","nonce":3471641477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":3471641477},{"type":"v1","addr":"172.21.15.177:6829","nonce":3471641477}]},"public_addr":"172.21.15.177:6823/3471641477","cluster_addr":"172.21.15.177:6827/3471641477","heartbeat_back_addr":"172.21.15.177:6831/3471641477","heartbeat_front_addr":"172.21.15.177:6829/3471641477","state":["exists","up"]},{"osd":2,"uuid":"c712ddbe-4915-4670-a683-4acdd11404e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6801","nonce":2694847262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6803","nonce":2694847262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6807","nonce":2694847262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2694847262},{"type":"v1","addr":"172.21.15.177:6805","nonce":2694847262}]},"public_addr":"172.21.15.177:6801/2694847262","cluster_addr":"172.21.15.177:6803/2694847262","heartbeat_back_addr":"172.21.15.177:6807/2694847262","heartbeat_front_addr":"172.21.15.177:6805/2694847262","state":["exists","up"]},{"osd":3,"uuid":"3e5c0168-06af-4e9a-8dd1-eb25895cb480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6817","nonce":4093771454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6819","nonce":4093771454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6826","nonce":4093771454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4093771454},{"type":"v1","addr":"172.21.15.177:6822","nonce":4093771454}]},"public_addr":"172.21.15.177:6817/4093771454","cluster_addr":"172.21.15.177:6819/4093771454","heartbeat_back_addr":"172.21.15.177:6826/4093771454","heartbeat_front_addr":"172.21.15.177:6822/4093771454","state":["exists","up"]},{"osd":4,"uuid":"6668ff75-4ff3-47a0-ba04-95ae6e233fe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6801","nonce":1585905529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6803","nonce":1585905529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6807","nonce":1585905529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1585905529},{"type":"v1","addr":"172.21.15.188:6805","nonce":1585905529}]},"public_addr":"172.21.15.188:6801/1585905529","cluster_addr":"172.21.15.188:6803/1585905529","heartbeat_back_addr":"172.21.15.188:6807/1585905529","heartbeat_front_addr":"172.21.15.188:6805/1585905529","state":["exists","up"]},{"osd":5,"uuid":"9e02c1b8-134f-43bf-917c-966ba4eac713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":58,"up_from":106,"up_thru":182,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6817","nonce":1469026711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6819","nonce":1469026711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6823","nonce":1469026711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1469026711},{"type":"v1","addr":"172.21.15.188:6821","nonce":1469026711}]},"public_addr":"172.21.15.188:6817/1469026711","cluster_addr":"172.21.15.188:6819/1469026711","heartbeat_back_addr":"172.21.15.188:6823/1469026711","heartbeat_front_addr":"172.21.15.188:6821/1469026711","state":["exists","up"]},{"osd":6,"uuid":"38f1ad8a-9e66-4e02-bf65-0d93e046ad5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6809","nonce":3787670496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6811","nonce":3787670496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6815","nonce":3787670496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3787670496},{"type":"v1","addr":"172.21.15.188:6813","nonce":3787670496}]},"public_addr":"172.21.15.188:6809/3787670496","cluster_addr":"172.21.15.188:6811/3787670496","heartbeat_back_addr":"172.21.15.188:6815/3787670496","heartbeat_front_addr":"172.21.15.188:6813/3787670496","state":["exists","up"]},{"osd":7,"uuid":"bd284e7d-47ec-489a-bd56-d5710e1fad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6825","nonce":405203122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6827","nonce":405203122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6831","nonce":405203122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":405203122},{"type":"v1","addr":"172.21.15.188:6829","nonce":405203122}]},"public_addr":"172.21.15.188:6825/405203122","cluster_addr":"172.21.15.188:6827/405203122","heartbeat_back_addr":"172.21.15.188:6831/405203122","heartbeat_front_addr":"172.21.15.188:6829/405203122","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.148758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.170231+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.129002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.179213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.075940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122695+0000","dead_epoch":61},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:10.421746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T03:19:08.122924+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.19","osds":[3,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T03:22:44.353 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T03:22:44.353 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:44.491 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:44.491 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:44.491 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:44.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T03:22:44.788 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T03:22:44.801 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T03:22:44.802 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:44.926 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:44.926 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:44.926 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:44.936 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T03:22:45.224 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T03:22:45.236 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T03:22:45.237 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:45.362 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:45.362 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:45.362 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:45.373 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T03:22:45.661 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T03:22:45.674 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T03:22:45.674 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:45.798 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:45.798 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:45.798 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:45.808 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T03:22:46.095 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T03:22:46.108 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T03:22:46.108 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:46.234 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:46.235 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:46.235 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:46.246 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T03:22:46.537 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T03:22:46.552 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T03:22:46.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:46.676 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:46.676 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:46.676 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:46.687 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T03:22:46.971 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T03:22:46.985 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T03:22:46.985 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:47.110 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:47.110 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:47.110 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:47.121 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T03:22:47.408 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T03:22:47.422 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T03:22:47.422 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T03:22:47.548 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-05-08T03:22:47.548 INFO:teuthology.orchestra.run.smithi177.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T03:22:47.548 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-05-08T03:22:47.562 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T03:22:47.847 INFO:teuthology.orchestra.run.smithi177.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T03:22:48.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:22:48.298 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:22:48.299 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:22:48.311 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-05-08T03:22:48.208697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20488,"ondisk_log_size":20488,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429304,"kb_used_data":19536,"kb_used_omap":19,"kb_used_meta":409644,"kb_avail":754545416,"statfs":{"total":773094113280,"available":772654505984,"internally_reserved":0,"allocated":20004864,"data_stored":10401774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":419475469},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.256031"},"pg_stats":[{"pgid":"3.1f","version":"176'390","reported_seq":3554,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497918+0000","last_change":"2024-05-08T03:22:26.808600+0000","last_active":"2024-05-08T03:22:38.497918+0000","last_peered":"2024-05-08T03:22:38.497918+0000","last_clean":"2024-05-08T03:22:38.497918+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:38.497918+0000","last_undegraded":"2024-05-08T03:22:38.497918+0000","last_fullsized":"2024-05-08T03:22:38.497918+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'356","reported_seq":3750,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509518+0000","last_change":"2024-05-08T03:22:26.803642+0000","last_active":"2024-05-08T03:22:38.509518+0000","last_peered":"2024-05-08T03:22:38.509518+0000","last_clean":"2024-05-08T03:22:38.509518+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:38.509518+0000","last_undegraded":"2024-05-08T03:22:38.509518+0000","last_fullsized":"2024-05-08T03:22:38.509518+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"173'542","reported_seq":7327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972369+0000","last_change":"2024-05-08T03:22:23.810462+0000","last_active":"2024-05-08T03:22:37.972369+0000","last_peered":"2024-05-08T03:22:37.972369+0000","last_clean":"2024-05-08T03:22:37.972369+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:37.972369+0000","last_undegraded":"2024-05-08T03:22:37.972369+0000","last_fullsized":"2024-05-08T03:22:37.972369+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:04:19.892753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'1045","reported_seq":4590,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972506+0000","last_change":"2024-05-08T03:22:23.810627+0000","last_active":"2024-05-08T03:22:37.972506+0000","last_peered":"2024-05-08T03:22:37.972506+0000","last_clean":"2024-05-08T03:22:37.972506+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:37.972506+0000","last_undegraded":"2024-05-08T03:22:37.972506+0000","last_fullsized":"2024-05-08T03:22:37.972506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:20:20.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'614","reported_seq":5216,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497860+0000","last_change":"2024-05-08T03:22:27.807218+0000","last_active":"2024-05-08T03:22:38.497860+0000","last_peered":"2024-05-08T03:22:38.497860+0000","last_clean":"2024-05-08T03:22:38.497860+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:38.497860+0000","last_undegraded":"2024-05-08T03:22:38.497860+0000","last_fullsized":"2024-05-08T03:22:38.497860+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"174'627","reported_seq":6212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882710+0000","last_change":"2024-05-08T03:22:16.868895+0000","last_active":"2024-05-08T03:22:38.882710+0000","last_peered":"2024-05-08T03:22:38.882710+0000","last_clean":"2024-05-08T03:22:38.882710+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:38.882710+0000","last_undegraded":"2024-05-08T03:22:38.882710+0000","last_fullsized":"2024-05-08T03:22:38.882710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:27.727588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'1161","reported_seq":4924,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509439+0000","last_change":"2024-05-08T03:22:26.805640+0000","last_active":"2024-05-08T03:22:38.509439+0000","last_peered":"2024-05-08T03:22:38.509439+0000","last_clean":"2024-05-08T03:22:38.509439+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:38.509439+0000","last_undegraded":"2024-05-08T03:22:38.509439+0000","last_fullsized":"2024-05-08T03:22:38.509439+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"176'632","reported_seq":10220,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509511+0000","last_change":"2024-05-08T03:22:27.804667+0000","last_active":"2024-05-08T03:22:38.509511+0000","last_peered":"2024-05-08T03:22:38.509511+0000","last_clean":"2024-05-08T03:22:38.509511+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:38.509511+0000","last_undegraded":"2024-05-08T03:22:38.509511+0000","last_fullsized":"2024-05-08T03:22:38.509511+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'422","reported_seq":5425,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497843+0000","last_change":"2024-05-08T03:22:24.798989+0000","last_active":"2024-05-08T03:22:38.497843+0000","last_peered":"2024-05-08T03:22:38.497843+0000","last_clean":"2024-05-08T03:22:38.497843+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:38.497843+0000","last_undegraded":"2024-05-08T03:22:38.497843+0000","last_fullsized":"2024-05-08T03:22:38.497843+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'296","reported_seq":3949,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972408+0000","last_change":"2024-05-08T03:22:23.808981+0000","last_active":"2024-05-08T03:22:37.972408+0000","last_peered":"2024-05-08T03:22:37.972408+0000","last_clean":"2024-05-08T03:22:37.972408+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:37.972408+0000","last_undegraded":"2024-05-08T03:22:37.972408+0000","last_fullsized":"2024-05-08T03:22:37.972408+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:57:01.587880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"175'435","reported_seq":8316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955970+0000","last_change":"2024-05-08T03:22:27.804313+0000","last_active":"2024-05-08T03:22:37.955970+0000","last_peered":"2024-05-08T03:22:37.955970+0000","last_clean":"2024-05-08T03:22:37.955970+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:37.955970+0000","last_undegraded":"2024-05-08T03:22:37.955970+0000","last_fullsized":"2024-05-08T03:22:37.955970+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:05:37.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"173'905","reported_seq":5646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882960+0000","last_change":"2024-05-08T03:22:32.932926+0000","last_active":"2024-05-08T03:22:38.882960+0000","last_peered":"2024-05-08T03:22:38.882960+0000","last_clean":"2024-05-08T03:22:38.882960+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:38.882960+0000","last_undegraded":"2024-05-08T03:22:38.882960+0000","last_fullsized":"2024-05-08T03:22:38.882960+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:43.327096+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196019+0000","last_change":"2024-05-08T03:22:36.072861+0000","last_active":"2024-05-08T03:22:38.196019+0000","last_peered":"2024-05-08T03:22:38.196019+0000","last_clean":"2024-05-08T03:22:38.196019+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:38.196019+0000","last_undegraded":"2024-05-08T03:22:38.196019+0000","last_fullsized":"2024-05-08T03:22:38.196019+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:42.430931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'455","reported_seq":4787,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972484+0000","last_change":"2024-05-08T03:22:25.810549+0000","last_active":"2024-05-08T03:22:37.972484+0000","last_peered":"2024-05-08T03:22:37.972484+0000","last_clean":"2024-05-08T03:22:37.972484+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:37.972484+0000","last_undegraded":"2024-05-08T03:22:37.972484+0000","last_fullsized":"2024-05-08T03:22:37.972484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:29:03.796868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.196031+0000","last_change":"2024-05-08T03:22:35.952124+0000","last_active":"2024-05-08T03:22:38.196031+0000","last_peered":"2024-05-08T03:22:38.196031+0000","last_clean":"2024-05-08T03:22:38.196031+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:38.196031+0000","last_undegraded":"2024-05-08T03:22:38.196031+0000","last_fullsized":"2024-05-08T03:22:38.196031+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:44.557317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"173'859","reported_seq":5902,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954186+0000","last_change":"2024-05-08T03:22:28.803724+0000","last_active":"2024-05-08T03:22:37.954186+0000","last_peered":"2024-05-08T03:22:37.954186+0000","last_clean":"2024-05-08T03:22:37.954186+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:22:37.954186+0000","last_undegraded":"2024-05-08T03:22:37.954186+0000","last_fullsized":"2024-05-08T03:22:37.954186+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":236,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882909+0000","last_change":"2024-05-08T03:20:34.778976+0000","last_active":"2024-05-08T03:22:38.882909+0000","last_peered":"2024-05-08T03:22:38.882909+0000","last_clean":"2024-05-08T03:22:38.882909+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:38.882909+0000","last_undegraded":"2024-05-08T03:22:38.882909+0000","last_fullsized":"2024-05-08T03:22:38.882909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:56:02.996181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"174'332","reported_seq":5755,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509573+0000","last_change":"2024-05-08T03:22:23.806536+0000","last_active":"2024-05-08T03:22:38.509573+0000","last_peered":"2024-05-08T03:22:38.509573+0000","last_clean":"2024-05-08T03:22:38.509573+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:38.509573+0000","last_undegraded":"2024-05-08T03:22:38.509573+0000","last_fullsized":"2024-05-08T03:22:38.509573+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509377+0000","last_change":"2024-05-08T03:22:21.828489+0000","last_active":"2024-05-08T03:22:38.509377+0000","last_peered":"2024-05-08T03:22:38.509377+0000","last_clean":"2024-05-08T03:22:38.509377+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:38.509377+0000","last_undegraded":"2024-05-08T03:22:38.509377+0000","last_fullsized":"2024-05-08T03:22:38.509377+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'1037","reported_seq":5279,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497918+0000","last_change":"2024-05-08T03:22:27.804984+0000","last_active":"2024-05-08T03:22:38.497918+0000","last_peered":"2024-05-08T03:22:38.497918+0000","last_clean":"2024-05-08T03:22:38.497918+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:38.497918+0000","last_undegraded":"2024-05-08T03:22:38.497918+0000","last_fullsized":"2024-05-08T03:22:38.497918+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"195'370","reported_seq":4543,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:45.327868+0000","last_change":"2024-05-08T03:22:45.327903+0000","last_active":"2024-05-08T03:22:45.327868+0000","last_peered":"2024-05-08T03:22:45.327868+0000","last_clean":"2024-05-08T03:22:45.327868+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:45.327868+0000","last_undegraded":"2024-05-08T03:22:45.327868+0000","last_fullsized":"2024-05-08T03:22:45.327868+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'370","last_scrub_stamp":"2024-05-08T03:22:45.327799+0000","last_deep_scrub":"195'370","last_deep_scrub_stamp":"2024-05-08T03:22:45.327799+0000","last_clean_scrub_stamp":"2024-05-08T03:22:45.327799+0000","objects_scrubbed":12,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:40.349336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":547,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955741+0000","last_change":"2024-05-08T03:22:28.803643+0000","last_active":"2024-05-08T03:22:37.955741+0000","last_peered":"2024-05-08T03:22:37.955741+0000","last_clean":"2024-05-08T03:22:37.955741+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:37.955741+0000","last_undegraded":"2024-05-08T03:22:37.955741+0000","last_fullsized":"2024-05-08T03:22:37.955741+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:12.719523+0000","last_clean_scrub_stamp":"2024-05-08T03:19:12.719523+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:46.908323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":378,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:45.526874+0000","last_change":"2024-05-08T03:22:45.526896+0000","last_active":"2024-05-08T03:22:45.526874+0000","last_peered":"2024-05-08T03:22:45.526874+0000","last_clean":"2024-05-08T03:22:45.526874+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:22:45.526874+0000","last_undegraded":"2024-05-08T03:22:45.526874+0000","last_fullsized":"2024-05-08T03:22:45.526874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:45.526801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:45.526801+0000","last_clean_scrub_stamp":"2024-05-08T03:22:45.526801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:31:06.401358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"174'520","reported_seq":7445,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.954151+0000","last_change":"2024-05-08T03:22:16.867571+0000","last_active":"2024-05-08T03:22:37.954151+0000","last_peered":"2024-05-08T03:22:37.954151+0000","last_clean":"2024-05-08T03:22:37.954151+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:22:37.954151+0000","last_undegraded":"2024-05-08T03:22:37.954151+0000","last_fullsized":"2024-05-08T03:22:37.954151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955724+0000","last_change":"2024-05-08T03:22:22.841205+0000","last_active":"2024-05-08T03:22:37.955724+0000","last_peered":"2024-05-08T03:22:37.955724+0000","last_clean":"2024-05-08T03:22:37.955724+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:37.955724+0000","last_undegraded":"2024-05-08T03:22:37.955724+0000","last_fullsized":"2024-05-08T03:22:37.955724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:22.701410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'1057","reported_seq":9575,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882835+0000","last_change":"2024-05-08T03:22:32.932833+0000","last_active":"2024-05-08T03:22:38.882835+0000","last_peered":"2024-05-08T03:22:38.882835+0000","last_clean":"2024-05-08T03:22:38.882835+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:38.882835+0000","last_undegraded":"2024-05-08T03:22:38.882835+0000","last_fullsized":"2024-05-08T03:22:38.882835+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:46:28.460551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972255+0000","last_change":"2024-05-08T03:22:22.839738+0000","last_active":"2024-05-08T03:22:37.972255+0000","last_peered":"2024-05-08T03:22:37.972255+0000","last_clean":"2024-05-08T03:22:37.972255+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:37.972255+0000","last_undegraded":"2024-05-08T03:22:37.972255+0000","last_fullsized":"2024-05-08T03:22:37.972255+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:59.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'922","reported_seq":10135,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509450+0000","last_change":"2024-05-08T03:22:26.806179+0000","last_active":"2024-05-08T03:22:38.509450+0000","last_peered":"2024-05-08T03:22:38.509450+0000","last_clean":"2024-05-08T03:22:38.509450+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:38.509450+0000","last_undegraded":"2024-05-08T03:22:38.509450+0000","last_fullsized":"2024-05-08T03:22:38.509450+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955748+0000","last_change":"2024-05-08T03:22:22.843526+0000","last_active":"2024-05-08T03:22:37.955748+0000","last_peered":"2024-05-08T03:22:37.955748+0000","last_clean":"2024-05-08T03:22:37.955748+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:37.955748+0000","last_undegraded":"2024-05-08T03:22:37.955748+0000","last_fullsized":"2024-05-08T03:22:37.955748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:11.717900+0000","last_clean_scrub_stamp":"2024-05-08T03:19:11.717900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:14.999728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'543","reported_seq":5934,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509414+0000","last_change":"2024-05-08T03:22:28.807411+0000","last_active":"2024-05-08T03:22:38.509414+0000","last_peered":"2024-05-08T03:22:38.509414+0000","last_clean":"2024-05-08T03:22:38.509414+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:38.509414+0000","last_undegraded":"2024-05-08T03:22:38.509414+0000","last_fullsized":"2024-05-08T03:22:38.509414+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'544","reported_seq":4060,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.882767+0000","last_change":"2024-05-08T03:22:16.871542+0000","last_active":"2024-05-08T03:22:38.882767+0000","last_peered":"2024-05-08T03:22:38.882767+0000","last_clean":"2024-05-08T03:22:38.882767+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:38.882767+0000","last_undegraded":"2024-05-08T03:22:38.882767+0000","last_fullsized":"2024-05-08T03:22:38.882767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:30.948114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"174'620","reported_seq":2412,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195885+0000","last_change":"2024-05-08T03:22:35.953685+0000","last_active":"2024-05-08T03:22:38.195885+0000","last_peered":"2024-05-08T03:22:38.195885+0000","last_clean":"2024-05-08T03:22:38.195885+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:38.195885+0000","last_undegraded":"2024-05-08T03:22:38.195885+0000","last_fullsized":"2024-05-08T03:22:38.195885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:13.225297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'479","reported_seq":7788,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.509463+0000","last_change":"2024-05-08T03:22:25.810877+0000","last_active":"2024-05-08T03:22:38.509463+0000","last_peered":"2024-05-08T03:22:38.509463+0000","last_clean":"2024-05-08T03:22:38.509463+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:38.509463+0000","last_undegraded":"2024-05-08T03:22:38.509463+0000","last_fullsized":"2024-05-08T03:22:38.509463+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"173'864","reported_seq":5489,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.972304+0000","last_change":"2024-05-08T03:22:25.811641+0000","last_active":"2024-05-08T03:22:37.972304+0000","last_peered":"2024-05-08T03:22:37.972304+0000","last_clean":"2024-05-08T03:22:37.972304+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:37.972304+0000","last_undegraded":"2024-05-08T03:22:37.972304+0000","last_fullsized":"2024-05-08T03:22:37.972304+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:29.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'1092","reported_seq":6035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955985+0000","last_change":"2024-05-08T03:22:24.800600+0000","last_active":"2024-05-08T03:22:37.955985+0000","last_peered":"2024-05-08T03:22:37.955985+0000","last_clean":"2024-05-08T03:22:37.955985+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:37.955985+0000","last_undegraded":"2024-05-08T03:22:37.955985+0000","last_fullsized":"2024-05-08T03:22:37.955985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:33.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"176'681","reported_seq":4829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955856+0000","last_change":"2024-05-08T03:22:16.876674+0000","last_active":"2024-05-08T03:22:37.955856+0000","last_peered":"2024-05-08T03:22:37.955856+0000","last_clean":"2024-05-08T03:22:37.955856+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:37.955856+0000","last_undegraded":"2024-05-08T03:22:37.955856+0000","last_fullsized":"2024-05-08T03:22:37.955856+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:32.370610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"164'612","reported_seq":4242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955904+0000","last_change":"2024-05-08T03:22:16.876490+0000","last_active":"2024-05-08T03:22:37.955904+0000","last_peered":"2024-05-08T03:22:37.955904+0000","last_clean":"2024-05-08T03:22:37.955904+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:37.955904+0000","last_undegraded":"2024-05-08T03:22:37.955904+0000","last_fullsized":"2024-05-08T03:22:37.955904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:15.091431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'522","reported_seq":5146,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.195845+0000","last_change":"2024-05-08T03:22:35.953206+0000","last_active":"2024-05-08T03:22:38.195845+0000","last_peered":"2024-05-08T03:22:38.195845+0000","last_clean":"2024-05-08T03:22:38.195845+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:38.195845+0000","last_undegraded":"2024-05-08T03:22:38.195845+0000","last_fullsized":"2024-05-08T03:22:38.195845+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:54.549900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'563","reported_seq":6395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955844+0000","last_change":"2024-05-08T03:22:16.874528+0000","last_active":"2024-05-08T03:22:37.955844+0000","last_peered":"2024-05-08T03:22:37.955844+0000","last_clean":"2024-05-08T03:22:37.955844+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:37.955844+0000","last_undegraded":"2024-05-08T03:22:37.955844+0000","last_fullsized":"2024-05-08T03:22:37.955844+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:22.149690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'430","reported_seq":3493,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:38.497787+0000","last_change":"2024-05-08T03:22:26.807535+0000","last_active":"2024-05-08T03:22:38.497787+0000","last_peered":"2024-05-08T03:22:38.497787+0000","last_clean":"2024-05-08T03:22:38.497787+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:38.497787+0000","last_undegraded":"2024-05-08T03:22:38.497787+0000","last_fullsized":"2024-05-08T03:22:38.497787+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'444","reported_seq":5637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:37.955764+0000","last_change":"2024-05-08T03:22:16.874743+0000","last_active":"2024-05-08T03:22:37.955764+0000","last_peered":"2024-05-08T03:22:37.955764+0000","last_clean":"2024-05-08T03:22:37.955764+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:37.955764+0000","last_undegraded":"2024-05-08T03:22:37.955764+0000","last_fullsized":"2024-05-08T03:22:37.955764+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:19:19.755678+0000","last_clean_scrub_stamp":"2024-05-08T03:19:19.755678+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:59:40.732000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20371,"ondisk_log_size":20371,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34060,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337780,"statfs":{"total":96636764160,"available":96601886720,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533399,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38344,"kb_used_data":2296,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333496,"statfs":{"total":96636764160,"available":96597499904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51524,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320316,"statfs":{"total":96636764160,"available":96584003584,"internally_reserved":0,"allocated":2355200,"data_stored":1152644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542193,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82636,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289204,"statfs":{"total":96636764160,"available":96552144896,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41800,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330040,"statfs":{"total":96636764160,"available":96593960960,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:22:48.316 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.316 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.316 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.316 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.316 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.316 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.317 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T03:19:12.719523+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=12, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.318 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T03:19:11.717900+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=11, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.319 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.320 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.320 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.320 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T03:19:19.755678+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=19, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=3, tm_min=22, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T03:22:48.320 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T03:23:08.321 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T03:23:08.607 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-05-08T03:23:08.608 INFO:teuthology.orchestra.run.smithi177.stderr:dumped all 2024-05-08T03:23:08.620 INFO:teuthology.orchestra.run.smithi177.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-05-08T03:23:08.212912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":607,"num_read":198953,"num_read_kb":1192475,"num_write":23906,"num_write_kb":750956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":687464325,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21089,"ondisk_log_size":21089,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":433424,"kb_used_data":19536,"kb_used_omap":19,"kb_used_meta":413804,"kb_avail":754541296,"statfs":{"total":773094113280,"available":772650287104,"internally_reserved":0,"allocated":20004864,"data_stored":10401774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20467,"internal_metadata":423735309},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001203"},"pg_stats":[{"pgid":"3.1f","version":"195'407","reported_seq":3579,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:48.319241+0000","last_change":"2024-05-08T03:22:48.319283+0000","last_active":"2024-05-08T03:22:48.319241+0000","last_peered":"2024-05-08T03:22:48.319241+0000","last_clean":"2024-05-08T03:22:48.319241+0000","last_became_active":"2024-05-08T03:22:26.806830+0000","last_became_peered":"2024-05-08T03:22:26.806830+0000","last_unstale":"2024-05-08T03:22:48.319241+0000","last_undegraded":"2024-05-08T03:22:48.319241+0000","last_fullsized":"2024-05-08T03:22:48.319241+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'407","last_scrub_stamp":"2024-05-08T03:22:48.319172+0000","last_deep_scrub":"195'407","last_deep_scrub_stamp":"2024-05-08T03:22:48.319172+0000","last_clean_scrub_stamp":"2024-05-08T03:22:48.319172+0000","objects_scrubbed":17,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:29:51.384334+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00088185100000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3073,"num_read_kb":31452,"num_write":410,"num_write_kb":17432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4005046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'381","reported_seq":3784,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:49.480234+0000","last_change":"2024-05-08T03:22:49.480288+0000","last_active":"2024-05-08T03:22:49.480234+0000","last_peered":"2024-05-08T03:22:49.480234+0000","last_clean":"2024-05-08T03:22:49.480234+0000","last_became_active":"2024-05-08T03:22:26.802006+0000","last_became_peered":"2024-05-08T03:22:26.802006+0000","last_unstale":"2024-05-08T03:22:49.480234+0000","last_undegraded":"2024-05-08T03:22:49.480234+0000","last_fullsized":"2024-05-08T03:22:49.480234+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'381","last_scrub_stamp":"2024-05-08T03:22:49.480135+0000","last_deep_scrub":"195'381","last_deep_scrub_stamp":"2024-05-08T03:22:49.480135+0000","last_clean_scrub_stamp":"2024-05-08T03:22:49.480135+0000","objects_scrubbed":25,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:59:03.319972+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0014041170000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3253,"num_read_kb":10717,"num_write":375,"num_write_kb":6619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":1675877,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"195'553","reported_seq":7347,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:49.960902+0000","last_change":"2024-05-08T03:22:49.960950+0000","last_active":"2024-05-08T03:22:49.960902+0000","last_peered":"2024-05-08T03:22:49.960902+0000","last_clean":"2024-05-08T03:22:49.960902+0000","last_became_active":"2024-05-08T03:22:23.808680+0000","last_became_peered":"2024-05-08T03:22:23.808680+0000","last_unstale":"2024-05-08T03:22:49.960902+0000","last_undegraded":"2024-05-08T03:22:49.960902+0000","last_fullsized":"2024-05-08T03:22:49.960902+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'553","last_scrub_stamp":"2024-05-08T03:22:49.960802+0000","last_deep_scrub":"195'553","last_deep_scrub_stamp":"2024-05-08T03:22:49.960802+0000","last_clean_scrub_stamp":"2024-05-08T03:22:49.960802+0000","objects_scrubbed":11,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:54:42.180827+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001684365,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7187,"num_read_kb":45592,"num_write":588,"num_write_kb":14725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":7329613,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"195'1062","reported_seq":4616,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.982276+0000","last_change":"2024-05-08T03:22:50.982351+0000","last_active":"2024-05-08T03:22:50.982276+0000","last_peered":"2024-05-08T03:22:50.982276+0000","last_clean":"2024-05-08T03:22:50.982276+0000","last_became_active":"2024-05-08T03:22:23.808768+0000","last_became_peered":"2024-05-08T03:22:23.808768+0000","last_unstale":"2024-05-08T03:22:50.982276+0000","last_undegraded":"2024-05-08T03:22:50.982276+0000","last_fullsized":"2024-05-08T03:22:50.982276+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1062","last_scrub_stamp":"2024-05-08T03:22:50.982133+0000","last_deep_scrub":"195'1062","last_deep_scrub_stamp":"2024-05-08T03:22:50.982133+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.982133+0000","objects_scrubbed":17,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:53:56.197748+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0021409530000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5966,"num_read_kb":52469,"num_write":1459,"num_write_kb":52330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":42493667,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"195'634","reported_seq":5244,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.340175+0000","last_change":"2024-05-08T03:22:50.340229+0000","last_active":"2024-05-08T03:22:50.340175+0000","last_peered":"2024-05-08T03:22:50.340175+0000","last_clean":"2024-05-08T03:22:50.340175+0000","last_became_active":"2024-05-08T03:22:27.806081+0000","last_became_peered":"2024-05-08T03:22:27.806081+0000","last_unstale":"2024-05-08T03:22:50.340175+0000","last_undegraded":"2024-05-08T03:22:50.340175+0000","last_fullsized":"2024-05-08T03:22:50.340175+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'634","last_scrub_stamp":"2024-05-08T03:22:50.340064+0000","last_deep_scrub":"195'634","last_deep_scrub_stamp":"2024-05-08T03:22:50.340064+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.340064+0000","objects_scrubbed":20,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:10:13.927336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0019361179999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6194,"num_read_kb":26697,"num_write":698,"num_write_kb":15971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":7523612,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"195'647","reported_seq":6241,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:49.055276+0000","last_change":"2024-05-08T03:22:49.055319+0000","last_active":"2024-05-08T03:22:49.055276+0000","last_peered":"2024-05-08T03:22:49.055276+0000","last_clean":"2024-05-08T03:22:49.055276+0000","last_became_active":"2024-05-08T03:20:39.773949+0000","last_became_peered":"2024-05-08T03:20:39.773949+0000","last_unstale":"2024-05-08T03:22:49.055276+0000","last_undegraded":"2024-05-08T03:22:49.055276+0000","last_fullsized":"2024-05-08T03:22:49.055276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'647","last_scrub_stamp":"2024-05-08T03:22:49.055198+0000","last_deep_scrub":"195'647","last_deep_scrub_stamp":"2024-05-08T03:22:49.055198+0000","last_clean_scrub_stamp":"2024-05-08T03:22:49.055198+0000","objects_scrubbed":20,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:58:21.044477+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.001068729,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":6085,"num_read_kb":45079,"num_write":811,"num_write_kb":52890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":36846023,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"195'1183","reported_seq":4954,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.466692+0000","last_change":"2024-05-08T03:22:50.466758+0000","last_active":"2024-05-08T03:22:50.466692+0000","last_peered":"2024-05-08T03:22:50.466692+0000","last_clean":"2024-05-08T03:22:50.466692+0000","last_became_active":"2024-05-08T03:22:26.803071+0000","last_became_peered":"2024-05-08T03:22:26.803071+0000","last_unstale":"2024-05-08T03:22:50.466692+0000","last_undegraded":"2024-05-08T03:22:50.466692+0000","last_fullsized":"2024-05-08T03:22:50.466692+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1183","last_scrub_stamp":"2024-05-08T03:22:50.466590+0000","last_deep_scrub":"195'1183","last_deep_scrub_stamp":"2024-05-08T03:22:50.466590+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.466590+0000","objects_scrubbed":22,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:12.387301+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.001826009,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7335,"num_read_kb":24642,"num_write":1803,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":6306969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"195'655","reported_seq":10251,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:52.389715+0000","last_change":"2024-05-08T03:22:52.389783+0000","last_active":"2024-05-08T03:22:52.389715+0000","last_peered":"2024-05-08T03:22:52.389715+0000","last_clean":"2024-05-08T03:22:52.389715+0000","last_became_active":"2024-05-08T03:22:27.802917+0000","last_became_peered":"2024-05-08T03:22:27.802917+0000","last_unstale":"2024-05-08T03:22:52.389715+0000","last_undegraded":"2024-05-08T03:22:52.389715+0000","last_fullsized":"2024-05-08T03:22:52.389715+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'655","last_scrub_stamp":"2024-05-08T03:22:52.389590+0000","last_deep_scrub":"195'655","last_deep_scrub_stamp":"2024-05-08T03:22:52.389590+0000","last_clean_scrub_stamp":"2024-05-08T03:22:52.389590+0000","objects_scrubbed":23,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:15:20.842005+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.0012635439999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":11373,"num_read_kb":61452,"num_write":583,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":24068759,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"195'438","reported_seq":5449,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:46.346123+0000","last_change":"2024-05-08T03:22:46.346159+0000","last_active":"2024-05-08T03:22:46.346123+0000","last_peered":"2024-05-08T03:22:46.346123+0000","last_clean":"2024-05-08T03:22:46.346123+0000","last_became_active":"2024-05-08T03:22:24.796795+0000","last_became_peered":"2024-05-08T03:22:24.796795+0000","last_unstale":"2024-05-08T03:22:46.346123+0000","last_undegraded":"2024-05-08T03:22:46.346123+0000","last_fullsized":"2024-05-08T03:22:46.346123+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'438","last_scrub_stamp":"2024-05-08T03:22:46.346038+0000","last_deep_scrub":"195'438","last_deep_scrub_stamp":"2024-05-08T03:22:46.346038+0000","last_clean_scrub_stamp":"2024-05-08T03:22:46.346038+0000","objects_scrubbed":16,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:06.488385+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0028634889999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5542,"num_read_kb":25702,"num_write":419,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":1250421,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"195'317","reported_seq":3979,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:48.968576+0000","last_change":"2024-05-08T03:22:48.968628+0000","last_active":"2024-05-08T03:22:48.968576+0000","last_peered":"2024-05-08T03:22:48.968576+0000","last_clean":"2024-05-08T03:22:48.968576+0000","last_became_active":"2024-05-08T03:22:23.807401+0000","last_became_peered":"2024-05-08T03:22:23.807401+0000","last_unstale":"2024-05-08T03:22:48.968576+0000","last_undegraded":"2024-05-08T03:22:48.968576+0000","last_fullsized":"2024-05-08T03:22:48.968576+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'317","last_scrub_stamp":"2024-05-08T03:22:48.968491+0000","last_deep_scrub":"195'317","last_deep_scrub_stamp":"2024-05-08T03:22:48.968491+0000","last_clean_scrub_stamp":"2024-05-08T03:22:48.968491+0000","objects_scrubbed":21,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:30.899001+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0029578149999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2599,"num_read_kb":61854,"num_write":255,"num_write_kb":45900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":62602978,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"195'459","reported_seq":8349,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:55.821830+0000","last_change":"2024-05-08T03:22:55.821893+0000","last_active":"2024-05-08T03:22:55.821830+0000","last_peered":"2024-05-08T03:22:55.821830+0000","last_clean":"2024-05-08T03:22:55.821830+0000","last_became_active":"2024-05-08T03:22:27.803078+0000","last_became_peered":"2024-05-08T03:22:27.803078+0000","last_unstale":"2024-05-08T03:22:55.821830+0000","last_undegraded":"2024-05-08T03:22:55.821830+0000","last_fullsized":"2024-05-08T03:22:55.821830+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'459","last_scrub_stamp":"2024-05-08T03:22:55.821744+0000","last_deep_scrub":"195'459","last_deep_scrub_stamp":"2024-05-08T03:22:55.821744+0000","last_clean_scrub_stamp":"2024-05-08T03:22:55.821744+0000","objects_scrubbed":24,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:33.502877+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0015560039999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7183,"num_read_kb":37750,"num_write":388,"num_write_kb":8173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":6918328,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"195'923","reported_seq":5674,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.997943+0000","last_change":"2024-05-08T03:22:50.997943+0000","last_active":"2024-05-08T03:22:50.997943+0000","last_peered":"2024-05-08T03:22:50.997943+0000","last_clean":"2024-05-08T03:22:50.997943+0000","last_became_active":"2024-05-08T03:22:32.931463+0000","last_became_peered":"2024-05-08T03:22:32.931463+0000","last_unstale":"2024-05-08T03:22:50.997943+0000","last_undegraded":"2024-05-08T03:22:50.997943+0000","last_fullsized":"2024-05-08T03:22:50.997943+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'923","last_scrub_stamp":"2024-05-08T03:22:50.997797+0000","last_deep_scrub":"195'923","last_deep_scrub_stamp":"2024-05-08T03:22:50.997797+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.997797+0000","objects_scrubbed":18,"log_size":923,"log_dups_size":0,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:05.832400+0000","scrub_duration":16,"objects_trimmed":4,"snaptrim_duration":0.027674771000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7619,"num_read_kb":22850,"num_write":1063,"num_write_kb":9506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":2580805,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":111,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.894292+0000","last_change":"2024-05-08T03:22:50.894313+0000","last_active":"2024-05-08T03:22:50.894292+0000","last_peered":"2024-05-08T03:22:50.894292+0000","last_clean":"2024-05-08T03:22:50.894292+0000","last_became_active":"2024-05-08T03:22:36.072612+0000","last_became_peered":"2024-05-08T03:22:36.072612+0000","last_unstale":"2024-05-08T03:22:50.894292+0000","last_undegraded":"2024-05-08T03:22:50.894292+0000","last_fullsized":"2024-05-08T03:22:50.894292+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:50.894163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:50.894163+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.894163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:49:40.596148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"195'476","reported_seq":4817,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:52.021547+0000","last_change":"2024-05-08T03:22:52.021598+0000","last_active":"2024-05-08T03:22:52.021547+0000","last_peered":"2024-05-08T03:22:52.021547+0000","last_clean":"2024-05-08T03:22:52.021547+0000","last_became_active":"2024-05-08T03:22:25.809263+0000","last_became_peered":"2024-05-08T03:22:25.809263+0000","last_unstale":"2024-05-08T03:22:52.021547+0000","last_undegraded":"2024-05-08T03:22:52.021547+0000","last_fullsized":"2024-05-08T03:22:52.021547+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'476","last_scrub_stamp":"2024-05-08T03:22:52.021461+0000","last_deep_scrub":"195'476","last_deep_scrub_stamp":"2024-05-08T03:22:52.021461+0000","last_clean_scrub_stamp":"2024-05-08T03:22:52.021461+0000","objects_scrubbed":21,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:19.432190+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.001071989,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4096,"num_read_kb":14199,"num_write":434,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3242087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:47.917948+0000","last_change":"2024-05-08T03:22:47.918005+0000","last_active":"2024-05-08T03:22:47.917948+0000","last_peered":"2024-05-08T03:22:47.917948+0000","last_clean":"2024-05-08T03:22:47.917948+0000","last_became_active":"2024-05-08T03:22:35.951797+0000","last_became_peered":"2024-05-08T03:22:35.951797+0000","last_unstale":"2024-05-08T03:22:47.917948+0000","last_undegraded":"2024-05-08T03:22:47.917948+0000","last_fullsized":"2024-05-08T03:22:47.917948+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:47.917849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:47.917849+0000","last_clean_scrub_stamp":"2024-05-08T03:22:47.917849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:28.281082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"195'882","reported_seq":5935,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-08T03:23:06.033721+0000","last_change":"2024-05-08T03:22:47.563980+0000","last_active":"2024-05-08T03:23:06.033721+0000","last_peered":"2024-05-08T03:23:06.033721+0000","last_clean":"2024-05-08T03:23:06.033721+0000","last_became_active":"2024-05-08T03:22:28.801697+0000","last_became_peered":"2024-05-08T03:22:28.801697+0000","last_unstale":"2024-05-08T03:23:06.033721+0000","last_undegraded":"2024-05-08T03:23:06.033721+0000","last_fullsized":"2024-05-08T03:23:06.033721+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'882","last_scrub_stamp":"2024-05-08T03:22:47.563816+0000","last_deep_scrub":"195'882","last_deep_scrub_stamp":"2024-05-08T03:22:47.563816+0000","last_clean_scrub_stamp":"2024-05-08T03:22:47.563816+0000","objects_scrubbed":23,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:20.014952+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.001644333,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7372,"num_read_kb":34343,"num_write":1021,"num_write_kb":18529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":12250278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":245,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:47.055342+0000","last_change":"2024-05-08T03:22:47.055367+0000","last_active":"2024-05-08T03:22:47.055342+0000","last_peered":"2024-05-08T03:22:47.055342+0000","last_clean":"2024-05-08T03:22:47.055342+0000","last_became_active":"2024-05-08T03:20:34.778793+0000","last_became_peered":"2024-05-08T03:20:34.778793+0000","last_unstale":"2024-05-08T03:22:47.055342+0000","last_undegraded":"2024-05-08T03:22:47.055342+0000","last_fullsized":"2024-05-08T03:22:47.055342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:47.055244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:47.055244+0000","last_clean_scrub_stamp":"2024-05-08T03:22:47.055244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:31.463571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"195'352","reported_seq":5783,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:47.514938+0000","last_change":"2024-05-08T03:22:47.515011+0000","last_active":"2024-05-08T03:22:47.514938+0000","last_peered":"2024-05-08T03:22:47.514938+0000","last_clean":"2024-05-08T03:22:47.514938+0000","last_became_active":"2024-05-08T03:22:23.804948+0000","last_became_peered":"2024-05-08T03:22:23.804948+0000","last_unstale":"2024-05-08T03:22:47.514938+0000","last_undegraded":"2024-05-08T03:22:47.514938+0000","last_fullsized":"2024-05-08T03:22:47.514938+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'352","last_scrub_stamp":"2024-05-08T03:22:47.514833+0000","last_deep_scrub":"195'352","last_deep_scrub_stamp":"2024-05-08T03:22:47.514833+0000","last_clean_scrub_stamp":"2024-05-08T03:22:47.514833+0000","objects_scrubbed":20,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:57:39.331949+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0021138720000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4527,"num_read_kb":31397,"num_write":337,"num_write_kb":15599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":13322692,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:46.499192+0000","last_change":"2024-05-08T03:22:46.499213+0000","last_active":"2024-05-08T03:22:46.499192+0000","last_peered":"2024-05-08T03:22:46.499192+0000","last_clean":"2024-05-08T03:22:46.499192+0000","last_became_active":"2024-05-08T03:22:21.828243+0000","last_became_peered":"2024-05-08T03:22:21.828243+0000","last_unstale":"2024-05-08T03:22:46.499192+0000","last_undegraded":"2024-05-08T03:22:46.499192+0000","last_fullsized":"2024-05-08T03:22:46.499192+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:46.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:46.499111+0000","last_clean_scrub_stamp":"2024-05-08T03:22:46.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:30:46.826094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"195'1058","reported_seq":5308,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:49.305654+0000","last_change":"2024-05-08T03:22:49.305717+0000","last_active":"2024-05-08T03:22:49.305654+0000","last_peered":"2024-05-08T03:22:49.305654+0000","last_clean":"2024-05-08T03:22:49.305654+0000","last_became_active":"2024-05-08T03:22:27.803045+0000","last_became_peered":"2024-05-08T03:22:27.803045+0000","last_unstale":"2024-05-08T03:22:49.305654+0000","last_undegraded":"2024-05-08T03:22:49.305654+0000","last_fullsized":"2024-05-08T03:22:49.305654+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1058","last_scrub_stamp":"2024-05-08T03:22:49.305568+0000","last_deep_scrub":"195'1058","last_deep_scrub_stamp":"2024-05-08T03:22:49.305568+0000","last_clean_scrub_stamp":"2024-05-08T03:22:49.305568+0000","objects_scrubbed":21,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:29.006162+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0020596780000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6098,"num_read_kb":34902,"num_write":1431,"num_write_kb":41086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":52332607,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"195'370","reported_seq":4543,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:45.327868+0000","last_change":"2024-05-08T03:22:45.327903+0000","last_active":"2024-05-08T03:22:45.327868+0000","last_peered":"2024-05-08T03:22:45.327868+0000","last_clean":"2024-05-08T03:22:45.327868+0000","last_became_active":"2024-05-08T03:22:24.796756+0000","last_became_peered":"2024-05-08T03:22:24.796756+0000","last_unstale":"2024-05-08T03:22:45.327868+0000","last_undegraded":"2024-05-08T03:22:45.327868+0000","last_fullsized":"2024-05-08T03:22:45.327868+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'370","last_scrub_stamp":"2024-05-08T03:22:45.327799+0000","last_deep_scrub":"195'370","last_deep_scrub_stamp":"2024-05-08T03:22:45.327799+0000","last_clean_scrub_stamp":"2024-05-08T03:22:45.327799+0000","objects_scrubbed":12,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:36:40.349336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026419479999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":2993,"num_read_kb":19741,"num_write":375,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":15872796,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"195'119","reported_seq":558,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:56.841209+0000","last_change":"2024-05-08T03:22:56.841242+0000","last_active":"2024-05-08T03:22:56.841209+0000","last_peered":"2024-05-08T03:22:56.841209+0000","last_clean":"2024-05-08T03:22:56.841209+0000","last_became_active":"2024-05-08T03:22:28.803326+0000","last_became_peered":"2024-05-08T03:22:28.803326+0000","last_unstale":"2024-05-08T03:22:56.841209+0000","last_undegraded":"2024-05-08T03:22:56.841209+0000","last_fullsized":"2024-05-08T03:22:56.841209+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2024-05-08T03:22:56.841123+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2024-05-08T03:22:56.841123+0000","last_clean_scrub_stamp":"2024-05-08T03:22:56.841123+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:41.221654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":380,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-08T03:23:06.033709+0000","last_change":"2024-05-08T03:22:45.526896+0000","last_active":"2024-05-08T03:23:06.033709+0000","last_peered":"2024-05-08T03:23:06.033709+0000","last_clean":"2024-05-08T03:23:06.033709+0000","last_became_active":"2024-05-08T03:19:12.731191+0000","last_became_peered":"2024-05-08T03:19:12.731191+0000","last_unstale":"2024-05-08T03:23:06.033709+0000","last_undegraded":"2024-05-08T03:23:06.033709+0000","last_fullsized":"2024-05-08T03:23:06.033709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:45.526801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:45.526801+0000","last_clean_scrub_stamp":"2024-05-08T03:22:45.526801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:31:06.401358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"195'537","reported_seq":7472,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-08T03:23:06.033824+0000","last_change":"2024-05-08T03:22:46.536773+0000","last_active":"2024-05-08T03:23:06.033824+0000","last_peered":"2024-05-08T03:23:06.033824+0000","last_clean":"2024-05-08T03:23:06.033824+0000","last_became_active":"2024-05-08T03:19:20.772938+0000","last_became_peered":"2024-05-08T03:19:20.772938+0000","last_unstale":"2024-05-08T03:23:06.033824+0000","last_undegraded":"2024-05-08T03:23:06.033824+0000","last_fullsized":"2024-05-08T03:23:06.033824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'537","last_scrub_stamp":"2024-05-08T03:22:46.536608+0000","last_deep_scrub":"195'537","last_deep_scrub_stamp":"2024-05-08T03:22:46.536608+0000","last_clean_scrub_stamp":"2024-05-08T03:22:46.536608+0000","objects_scrubbed":17,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:33.064229+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0020171619999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7457,"num_read_kb":29508,"num_write":498,"num_write_kb":32558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":92,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:52.805951+0000","last_change":"2024-05-08T03:22:52.805999+0000","last_active":"2024-05-08T03:22:52.805951+0000","last_peered":"2024-05-08T03:22:52.805951+0000","last_clean":"2024-05-08T03:22:52.805951+0000","last_became_active":"2024-05-08T03:22:22.840742+0000","last_became_peered":"2024-05-08T03:22:22.840742+0000","last_unstale":"2024-05-08T03:22:52.805951+0000","last_undegraded":"2024-05-08T03:22:52.805951+0000","last_fullsized":"2024-05-08T03:22:52.805951+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:52.805825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:52.805825+0000","last_clean_scrub_stamp":"2024-05-08T03:22:52.805825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:44:28.227526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"195'1074","reported_seq":9601,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.031120+0000","last_change":"2024-05-08T03:22:50.031173+0000","last_active":"2024-05-08T03:22:50.031120+0000","last_peered":"2024-05-08T03:22:50.031120+0000","last_clean":"2024-05-08T03:22:50.031120+0000","last_became_active":"2024-05-08T03:22:32.931453+0000","last_became_peered":"2024-05-08T03:22:32.931453+0000","last_unstale":"2024-05-08T03:22:50.031120+0000","last_undegraded":"2024-05-08T03:22:50.031120+0000","last_fullsized":"2024-05-08T03:22:50.031120+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1074","last_scrub_stamp":"2024-05-08T03:22:50.031011+0000","last_deep_scrub":"195'1074","last_deep_scrub_stamp":"2024-05-08T03:22:50.031011+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.031011+0000","objects_scrubbed":17,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:21.353084+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.003435764,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11884,"num_read_kb":56073,"num_write":1236,"num_write_kb":28151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":12533848,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:47.984997+0000","last_change":"2024-05-08T03:22:47.985033+0000","last_active":"2024-05-08T03:22:47.984997+0000","last_peered":"2024-05-08T03:22:47.984997+0000","last_clean":"2024-05-08T03:22:47.984997+0000","last_became_active":"2024-05-08T03:22:22.839455+0000","last_became_peered":"2024-05-08T03:22:22.839455+0000","last_unstale":"2024-05-08T03:22:47.984997+0000","last_undegraded":"2024-05-08T03:22:47.984997+0000","last_fullsized":"2024-05-08T03:22:47.984997+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:47.984894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:47.984894+0000","last_clean_scrub_stamp":"2024-05-08T03:22:47.984894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:13.063539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"195'940","reported_seq":10161,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:51.421130+0000","last_change":"2024-05-08T03:22:51.421224+0000","last_active":"2024-05-08T03:22:51.421130+0000","last_peered":"2024-05-08T03:22:51.421130+0000","last_clean":"2024-05-08T03:22:51.421130+0000","last_became_active":"2024-05-08T03:22:26.804012+0000","last_became_peered":"2024-05-08T03:22:26.804012+0000","last_unstale":"2024-05-08T03:22:51.421130+0000","last_undegraded":"2024-05-08T03:22:51.421130+0000","last_fullsized":"2024-05-08T03:22:51.421130+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'940","last_scrub_stamp":"2024-05-08T03:22:51.421009+0000","last_deep_scrub":"195'940","last_deep_scrub_stamp":"2024-05-08T03:22:51.421009+0000","last_clean_scrub_stamp":"2024-05-08T03:22:51.421009+0000","objects_scrubbed":18,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:05:52.333600+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0013453549999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":11268,"num_read_kb":56254,"num_write":1185,"num_write_kb":17007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15385325,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:53.775137+0000","last_change":"2024-05-08T03:22:53.775164+0000","last_active":"2024-05-08T03:22:53.775137+0000","last_peered":"2024-05-08T03:22:53.775137+0000","last_clean":"2024-05-08T03:22:53.775137+0000","last_became_active":"2024-05-08T03:22:22.843237+0000","last_became_peered":"2024-05-08T03:22:22.843237+0000","last_unstale":"2024-05-08T03:22:53.775137+0000","last_undegraded":"2024-05-08T03:22:53.775137+0000","last_fullsized":"2024-05-08T03:22:53.775137+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T03:22:53.775043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T03:22:53.775043+0000","last_clean_scrub_stamp":"2024-05-08T03:22:53.775043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:43.413612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"195'561","reported_seq":5960,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:53.389517+0000","last_change":"2024-05-08T03:22:53.389567+0000","last_active":"2024-05-08T03:22:53.389517+0000","last_peered":"2024-05-08T03:22:53.389517+0000","last_clean":"2024-05-08T03:22:53.389517+0000","last_became_active":"2024-05-08T03:22:28.806017+0000","last_became_peered":"2024-05-08T03:22:28.806017+0000","last_unstale":"2024-05-08T03:22:53.389517+0000","last_undegraded":"2024-05-08T03:22:53.389517+0000","last_fullsized":"2024-05-08T03:22:53.389517+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'561","last_scrub_stamp":"2024-05-08T03:22:53.389435+0000","last_deep_scrub":"195'561","last_deep_scrub_stamp":"2024-05-08T03:22:53.389435+0000","last_clean_scrub_stamp":"2024-05-08T03:22:53.389435+0000","objects_scrubbed":18,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:24.172078+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0013570909999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":7097,"num_read_kb":17892,"num_write":505,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":5273250,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"195'564","reported_seq":4090,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:48.082026+0000","last_change":"2024-05-08T03:22:48.082026+0000","last_active":"2024-05-08T03:22:48.082026+0000","last_peered":"2024-05-08T03:22:48.082026+0000","last_clean":"2024-05-08T03:22:48.082026+0000","last_became_active":"2024-05-08T03:20:37.783894+0000","last_became_peered":"2024-05-08T03:20:37.783894+0000","last_unstale":"2024-05-08T03:22:48.082026+0000","last_undegraded":"2024-05-08T03:22:48.082026+0000","last_fullsized":"2024-05-08T03:22:48.082026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'564","last_scrub_stamp":"2024-05-08T03:22:48.081844+0000","last_deep_scrub":"195'564","last_deep_scrub_stamp":"2024-05-08T03:22:48.081844+0000","last_clean_scrub_stamp":"2024-05-08T03:22:48.081844+0000","objects_scrubbed":20,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:52.303949+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0032817390000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4257,"num_read_kb":26399,"num_write":666,"num_write_kb":22947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":14203950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"195'634","reported_seq":2435,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:49.886727+0000","last_change":"2024-05-08T03:22:49.886776+0000","last_active":"2024-05-08T03:22:49.886727+0000","last_peered":"2024-05-08T03:22:49.886727+0000","last_clean":"2024-05-08T03:22:49.886727+0000","last_became_active":"2024-05-08T03:22:35.952601+0000","last_became_peered":"2024-05-08T03:22:35.952601+0000","last_unstale":"2024-05-08T03:22:49.886727+0000","last_undegraded":"2024-05-08T03:22:49.886727+0000","last_fullsized":"2024-05-08T03:22:49.886727+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'634","last_scrub_stamp":"2024-05-08T03:22:49.886640+0000","last_deep_scrub":"195'634","last_deep_scrub_stamp":"2024-05-08T03:22:49.886640+0000","last_clean_scrub_stamp":"2024-05-08T03:22:49.886640+0000","objects_scrubbed":14,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:38:44.549606+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0041625919999999997,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3234,"num_read_kb":10428,"num_write":658,"num_write_kb":5639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":4142836,"num_keys_recovered":39,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"195'500","reported_seq":7817,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:48.499759+0000","last_change":"2024-05-08T03:22:48.499871+0000","last_active":"2024-05-08T03:22:48.499759+0000","last_peered":"2024-05-08T03:22:48.499759+0000","last_clean":"2024-05-08T03:22:48.499759+0000","last_became_active":"2024-05-08T03:22:25.808674+0000","last_became_peered":"2024-05-08T03:22:25.808674+0000","last_unstale":"2024-05-08T03:22:48.499759+0000","last_undegraded":"2024-05-08T03:22:48.499759+0000","last_fullsized":"2024-05-08T03:22:48.499759+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'500","last_scrub_stamp":"2024-05-08T03:22:48.499671+0000","last_deep_scrub":"195'500","last_deep_scrub_stamp":"2024-05-08T03:22:48.499671+0000","last_clean_scrub_stamp":"2024-05-08T03:22:48.499671+0000","objects_scrubbed":21,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:58.955243+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002916713,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6943,"num_read_kb":59043,"num_write":485,"num_write_kb":18522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":37574118,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"195'890","reported_seq":5525,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:53.042599+0000","last_change":"2024-05-08T03:22:53.042627+0000","last_active":"2024-05-08T03:22:53.042599+0000","last_peered":"2024-05-08T03:22:53.042599+0000","last_clean":"2024-05-08T03:22:53.042599+0000","last_became_active":"2024-05-08T03:22:25.810104+0000","last_became_peered":"2024-05-08T03:22:25.810104+0000","last_unstale":"2024-05-08T03:22:53.042599+0000","last_undegraded":"2024-05-08T03:22:53.042599+0000","last_fullsized":"2024-05-08T03:22:53.042599+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'890","last_scrub_stamp":"2024-05-08T03:22:53.042512+0000","last_deep_scrub":"195'890","last_deep_scrub_stamp":"2024-05-08T03:22:53.042512+0000","last_clean_scrub_stamp":"2024-05-08T03:22:53.042512+0000","objects_scrubbed":26,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:54.879386+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.0027170290000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7272,"num_read_kb":23850,"num_write":1097,"num_write_kb":23638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":16989089,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"195'1113","reported_seq":6065,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:54.827373+0000","last_change":"2024-05-08T03:22:54.827471+0000","last_active":"2024-05-08T03:22:54.827373+0000","last_peered":"2024-05-08T03:22:54.827373+0000","last_clean":"2024-05-08T03:22:54.827373+0000","last_became_active":"2024-05-08T03:22:24.798961+0000","last_became_peered":"2024-05-08T03:22:24.798961+0000","last_unstale":"2024-05-08T03:22:54.827373+0000","last_undegraded":"2024-05-08T03:22:54.827373+0000","last_fullsized":"2024-05-08T03:22:54.827373+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1113","last_scrub_stamp":"2024-05-08T03:22:54.827269+0000","last_deep_scrub":"195'1113","last_deep_scrub_stamp":"2024-05-08T03:22:54.827269+0000","last_clean_scrub_stamp":"2024-05-08T03:22:54.827269+0000","objects_scrubbed":21,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T03:37:41.301720+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0016260210000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9433,"num_read_kb":75249,"num_write":1374,"num_write_kb":49375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":49256138,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"195'697","reported_seq":4854,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:51.837132+0000","last_change":"2024-05-08T03:22:51.837189+0000","last_active":"2024-05-08T03:22:51.837132+0000","last_peered":"2024-05-08T03:22:51.837132+0000","last_clean":"2024-05-08T03:22:51.837132+0000","last_became_active":"2024-05-08T03:22:05.795088+0000","last_became_peered":"2024-05-08T03:22:05.795088+0000","last_unstale":"2024-05-08T03:22:51.837132+0000","last_undegraded":"2024-05-08T03:22:51.837132+0000","last_fullsized":"2024-05-08T03:22:51.837132+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'697","last_scrub_stamp":"2024-05-08T03:22:51.837044+0000","last_deep_scrub":"195'697","last_deep_scrub_stamp":"2024-05-08T03:22:51.837044+0000","last_clean_scrub_stamp":"2024-05-08T03:22:51.837044+0000","objects_scrubbed":16,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:23.853562+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0020871919999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4693,"num_read_kb":56984,"num_write":851,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":59495044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"195'631","reported_seq":4270,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:50.792990+0000","last_change":"2024-05-08T03:22:50.793049+0000","last_active":"2024-05-08T03:22:50.792990+0000","last_peered":"2024-05-08T03:22:50.792990+0000","last_clean":"2024-05-08T03:22:50.792990+0000","last_became_active":"2024-05-08T03:22:04.797029+0000","last_became_peered":"2024-05-08T03:22:04.797029+0000","last_unstale":"2024-05-08T03:22:50.792990+0000","last_undegraded":"2024-05-08T03:22:50.792990+0000","last_fullsized":"2024-05-08T03:22:50.792990+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'631","last_scrub_stamp":"2024-05-08T03:22:50.792892+0000","last_deep_scrub":"195'631","last_deep_scrub_stamp":"2024-05-08T03:22:50.792892+0000","last_clean_scrub_stamp":"2024-05-08T03:22:50.792892+0000","objects_scrubbed":19,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:49:26.062741+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0019033800000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4010,"num_read_kb":15824,"num_write":773,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":11092817,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"195'545","reported_seq":5178,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:48.938877+0000","last_change":"2024-05-08T03:22:48.938931+0000","last_active":"2024-05-08T03:22:48.938877+0000","last_peered":"2024-05-08T03:22:48.938877+0000","last_clean":"2024-05-08T03:22:48.938877+0000","last_became_active":"2024-05-08T03:22:35.952038+0000","last_became_peered":"2024-05-08T03:22:35.952038+0000","last_unstale":"2024-05-08T03:22:48.938877+0000","last_undegraded":"2024-05-08T03:22:48.938877+0000","last_fullsized":"2024-05-08T03:22:48.938877+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'545","last_scrub_stamp":"2024-05-08T03:22:48.938798+0000","last_deep_scrub":"195'545","last_deep_scrub_stamp":"2024-05-08T03:22:48.938798+0000","last_clean_scrub_stamp":"2024-05-08T03:22:48.938798+0000","objects_scrubbed":23,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:16.917720+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0042537510000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6002,"num_read_kb":38235,"num_write":481,"num_write_kb":18998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":26805518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"195'576","reported_seq":6417,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:49.812359+0000","last_change":"2024-05-08T03:22:49.812407+0000","last_active":"2024-05-08T03:22:49.812359+0000","last_peered":"2024-05-08T03:22:49.812359+0000","last_clean":"2024-05-08T03:22:49.812359+0000","last_became_active":"2024-05-08T03:22:03.812685+0000","last_became_peered":"2024-05-08T03:22:03.812685+0000","last_unstale":"2024-05-08T03:22:49.812359+0000","last_undegraded":"2024-05-08T03:22:49.812359+0000","last_fullsized":"2024-05-08T03:22:49.812359+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'576","last_scrub_stamp":"2024-05-08T03:22:49.812275+0000","last_deep_scrub":"195'576","last_deep_scrub_stamp":"2024-05-08T03:22:49.812275+0000","last_clean_scrub_stamp":"2024-05-08T03:22:49.812275+0000","objects_scrubbed":13,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:02:42.728203+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00051206899999999998,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7343,"num_read_kb":66285,"num_write":509,"num_write_kb":56647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":66727905,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"195'446","reported_seq":3517,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:47.311875+0000","last_change":"2024-05-08T03:22:47.311922+0000","last_active":"2024-05-08T03:22:47.311875+0000","last_peered":"2024-05-08T03:22:47.311875+0000","last_clean":"2024-05-08T03:22:47.311875+0000","last_became_active":"2024-05-08T03:22:26.805726+0000","last_became_peered":"2024-05-08T03:22:26.805726+0000","last_unstale":"2024-05-08T03:22:47.311875+0000","last_undegraded":"2024-05-08T03:22:47.311875+0000","last_fullsized":"2024-05-08T03:22:47.311875+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'446","last_scrub_stamp":"2024-05-08T03:22:47.311801+0000","last_deep_scrub":"195'446","last_deep_scrub_stamp":"2024-05-08T03:22:47.311801+0000","last_clean_scrub_stamp":"2024-05-08T03:22:47.311801+0000","objects_scrubbed":16,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:56.397793+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00114832,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4479,"num_read_kb":30232,"num_write":431,"num_write_kb":31586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":9397402,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"195'465","reported_seq":5667,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-08T03:22:48.860714+0000","last_change":"2024-05-08T03:22:48.860775+0000","last_active":"2024-05-08T03:22:48.860714+0000","last_peered":"2024-05-08T03:22:48.860714+0000","last_clean":"2024-05-08T03:22:48.860714+0000","last_became_active":"2024-05-08T03:22:03.813163+0000","last_became_peered":"2024-05-08T03:22:03.813163+0000","last_unstale":"2024-05-08T03:22:48.860714+0000","last_undegraded":"2024-05-08T03:22:48.860714+0000","last_fullsized":"2024-05-08T03:22:48.860714+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'465","last_scrub_stamp":"2024-05-08T03:22:48.860630+0000","last_deep_scrub":"195'465","last_deep_scrub_stamp":"2024-05-08T03:22:48.860630+0000","last_clean_scrub_stamp":"2024-05-08T03:22:48.860630+0000","objects_scrubbed":21,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:04.263435+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.00065139899999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4970,"num_read_kb":49281,"num_write":548,"num_write_kb":27661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":53831951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":198837,"num_read_kb":1192375,"num_write":23747,"num_write_kb":748540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9248,"num_bytes_recovered":683331749,"num_keys_recovered":782,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},"log_size":20970,"ondisk_log_size":20970,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":5},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34052,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337788,"statfs":{"total":96636764160,"available":96601894912,"internally_reserved":0,"allocated":2945024,"data_stored":1742993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73728,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298112,"statfs":{"total":96636764160,"available":96561266688,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":106,"seq":455266533403,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38336,"kb_used_data":2296,"kb_used_omap":9,"kb_used_meta":36022,"kb_avail":94333504,"statfs":{"total":96636764160,"available":96597508096,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9310,"internal_metadata":36887458},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51524,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320316,"statfs":{"total":96636764160,"available":96584003584,"internally_reserved":0,"allocated":2355200,"data_stored":1152644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47360,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324480,"statfs":{"total":96636764160,"available":96588267520,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59840,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312000,"statfs":{"total":96636764160,"available":96575488000,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82632,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289208,"statfs":{"total":96636764160,"available":96552148992,"internally_reserved":0,"allocated":2949120,"data_stored":1743012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45952,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325888,"statfs":{"total":96636764160,"available":96589709312,"internally_reserved":0,"allocated":2351104,"data_stored":1152625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4028,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T03:23:08.623 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T03:23:08.925 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T03:23:08.925 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T03:23:08.925 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T03:23:08.925 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:08.960 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T03:23:08.960 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T03:23:08.960 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.031 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T03:23:09.032 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T03:23:09.032 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.107 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T03:23:09.107 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T03:23:09.107 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.176 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T03:23:09.177 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T03:23:09.177 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.213 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T03:23:09.214 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T03:23:09.214 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.286 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T03:23:09.286 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T03:23:09.286 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.367 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T03:23:09.368 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T03:23:09.368 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.405 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T03:23:09.406 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T03:23:09.406 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T03:23:09.406 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.432 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T03:23:09.432 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T03:23:09.432 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.472 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T03:23:09.473 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T03:23:09.473 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T03:23:09.473 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.485 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T03:23:09.485 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T03:23:09.485 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.537 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T03:23:09.537 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T03:23:09.538 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T03:23:09.549 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T03:23:09.549 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T03:23:09.549 DEBUG:teuthology.orchestra.run.smithi177:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-05-08T03:23:09.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:23:09.612 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T03:23:09.896 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:23:09.896 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T03:23:09.973 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:23:09.973 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T03:23:10.029 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:23:10.030 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T03:23:10.086 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:23:10.087 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T03:23:10.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:23:10.238 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T03:23:10.315 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:23:10.316 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T03:23:10.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:23:10.374 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T03:23:10.430 INFO:tasks.ceph:Archiving mon data... 2024-05-08T03:23:10.431 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/data/mon.a.tgz 2024-05-08T03:23:10.431 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-05-08T03:23:10.447 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.jb2rock4YG 2024-05-08T03:23:10.447 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.jb2rock4YG 2024-05-08T03:23:10.628 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.jb2rock4YG 2024-05-08T03:23:10.711 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.jb2rock4YG is 409KB 2024-05-08T03:23:10.781 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.jb2rock4YG 2024-05-08T03:23:10.797 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/data/mon.c.tgz 2024-05-08T03:23:10.797 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-05-08T03:23:10.852 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.Uy69X5lwxn 2024-05-08T03:23:10.852 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Uy69X5lwxn 2024-05-08T03:23:11.029 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.Uy69X5lwxn 2024-05-08T03:23:11.112 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.Uy69X5lwxn is 552KB 2024-05-08T03:23:11.182 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.Uy69X5lwxn 2024-05-08T03:23:11.197 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/data/mon.b.tgz 2024-05-08T03:23:11.198 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-05-08T03:23:11.216 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.LAz4beTp2X 2024-05-08T03:23:11.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LAz4beTp2X 2024-05-08T03:23:11.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.LAz4beTp2X 2024-05-08T03:23:11.494 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.LAz4beTp2X is 552KB 2024-05-08T03:23:11.593 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.LAz4beTp2X 2024-05-08T03:23:11.608 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T03:23:11.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T03:23:11.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T03:23:11.689 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T03:23:11.728 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T03:23:11.728 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T03:23:11.729 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/remote/smithi177/crash 2024-05-08T03:23:11.730 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T03:23:11.755 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/remote/smithi188/crash 2024-05-08T03:23:11.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T03:23:11.792 INFO:tasks.ceph:Compressing logs... 2024-05-08T03:23:11.792 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T03:23:11.800 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T03:23:11.822 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50076.log 2024-05-08T03:23:11.822 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2024-05-08T03:23:11.885 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T03:23:11.885 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32445.log 2024-05-08T03:23:11.885 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T03:23:11.885 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.32445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32405.log 2024-05-08T03:23:11.885 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32445.log.gz 2024-05-08T03:23:11.885 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32405.log.gz 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32576.log 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32365.log 2024-05-08T03:23:11.886 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32576.log.gz 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32365.log.gz 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5/var/log/ceph/ceph-mgr.x.log: --verbose -- /var/log/ceph/ceph-client.admin.32656.log 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32485.log 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32656.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32656.log.gz 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32616.log 2024-05-08T03:23:11.887 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32485.log.gz 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32527.log 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32616.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32616.log.gz 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr: --verbose -- /var/log/ceph/objectstore_tool.41501.log 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.32527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32527.log.gz 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/objectstore_tool.41501.log: /var/log/ceph/ceph-mon.b.log: 93.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T03:23:11.888 INFO:teuthology.orchestra.run.smithi188.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.41501.log.gz 2024-05-08T03:23:11.977 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.50076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61155.log 2024-05-08T03:23:11.977 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50076.log.gz 2024-05-08T03:23:11.977 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2024-05-08T03:23:11.977 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2024-05-08T03:23:11.977 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44769.log 2024-05-08T03:23:11.977 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61155.log.gz 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41819.log 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44769.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44769.log.gz -- 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr: /var/log/ceph/ceph-client.admin.57738.log 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58832.log 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41819.log.gz 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62097.log 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57738.log.gz 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61453.log 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58832.log.gz 2024-05-08T03:23:11.978 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54357.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62097.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62015.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61453.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38481.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54357.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62015.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54505.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38481.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59412.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43395.log 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54505.log.gz 2024-05-08T03:23:11.979 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40958.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59412.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32607.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63255.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40958.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57432.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.32607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32607.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39401.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63255.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57617.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57432.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39401.log.gz 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2024-05-08T03:23:11.980 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57617.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59850.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46728.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59850.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57917.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50750.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46728.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37796.log 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57917.log.gz 2024-05-08T03:23:11.981 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.50750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53455.log 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62583.log 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37796.log.gz 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53455.log.gz 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55559.log 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62583.log.gz 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40139.log 2024-05-08T03:23:11.982 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55559.log.gz 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40139.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40139.log.gz 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.50911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60903.log 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-05-08T03:23:11.983 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40519.log 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41297.log 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60903.log.gz 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61454.log 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40519.log.gz 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41297.log.gz 2024-05-08T03:23:11.984 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51645.log 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61454.log.gz 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60310.log 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51645.log.gz 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60310.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60310.log.gz 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40001.log 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37612.log 2024-05-08T03:23:11.985 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39399.log 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40001.log.gz 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37612.log.gz 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39031.log 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39399.log.gz 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.49505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52727.log 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46190.log 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39031.log.gz 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47458.log 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52727.log.gz 2024-05-08T03:23:11.986 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60478.log 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46190.log.gz 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56338.log 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47458.log.gz 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62373.log 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60478.log.gz 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38687.log 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56338.log.gz 2024-05-08T03:23:11.987 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45349.log 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62373.log.gz 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49245.log 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38687.log.gz 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60567.log 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45349.log.gz 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.49245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57274.log 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49245.log.gz 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63339.log 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60567.log.gz 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57796.log 2024-05-08T03:23:11.988 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57274.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61679.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63339.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48350.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57796.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61759.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61679.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58675.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48350.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60520.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61759.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58675.log.gz 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55781.log 2024-05-08T03:23:11.989 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60520.log.gz 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58573.log 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61239.log 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55781.log.gz 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58573.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58573.log.gz -5 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40577.log 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52431.log 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61239.log.gz 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40577.log.gz 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52431.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52431.log.gz -5 2024-05-08T03:23:11.990 INFO:teuthology.orchestra.run.smithi177.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51501.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.50318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39314.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51501.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55700.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.0.39314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61754.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32488.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55700.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.39314.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62096.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61754.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.32488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60252.log 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32488.log.gz 2024-05-08T03:23:11.991 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51884.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62096.log.gz 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60252.log.gz 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51884.log.gz 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62095.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49009.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62415.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62095.log.gz 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.49009.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.49009.log.gz /var/log/ceph/ceph-osd.3.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47299.log 2024-05-08T03:23:11.992 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62415.log.gz 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61811.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51107.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47299.log.gz 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61811.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.61811.log.gz /var/log/ceph/ceph-client.admin.47200.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44891.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51107.log.gz 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52111.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47200.log.gz 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38690.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44891.log.gz 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44334.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52111.log.gz 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-05-08T03:23:11.993 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38690.log.gz 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44334.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44334.log.gz --verbose 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-client.admin.38013.log 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54823.log 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38013.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38013.log.gz 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54823.log.gz 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63379.log 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63003.log 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz 2024-05-08T03:23:11.994 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63379.log.gzgzip 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41438.log 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38989.log 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63003.log.gz 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37668.log 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41438.log.gz 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38989.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.38989.log.gz --verbose 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-client.admin.41396.log 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59274.log 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37668.log.gz 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41396.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42234.log -- replaced with /var/log/ceph/ceph-client.admin.41396.log.gz 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59274.log.gz 2024-05-08T03:23:11.995 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42234.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42234.log.gz 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49841.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44491.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.49841.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.49841.log.gz -5 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43894.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44491.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44491.log.gz 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56713.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43894.log.gz 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56713.log.gz 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38689.log 2024-05-08T03:23:11.996 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47892.log 2024-05-08T03:23:11.997 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38689.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38689.log.gz 2024-05-08T03:23:11.997 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48134.log 2024-05-08T03:23:11.997 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59773.log 2024-05-08T03:23:11.997 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47892.log.gz 2024-05-08T03:23:11.997 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48134.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48134.log.gz 2024-05-08T03:23:11.997 INFO:teuthology.orchestra.run.smithi177.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53933.log 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45493.log 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59773.log.gz 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53933.log.gz 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61675.log 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45493.log.gz 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-05-08T03:23:11.998 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61675.log.gz 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62157.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62157.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62157.log.gz 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52938.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61995.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52938.log.gz 2024-05-08T03:23:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61995.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50036.log: -- /var/log/ceph/ceph-client.admin.50453.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44172.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50453.log: -- /var/log/ceph/ceph-client.admin.47498.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44172.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48691.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5/var/log/ceph/ceph-client.admin.47498.log: --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47498.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48691.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48691.log.gz 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42078.log 2024-05-08T03:23:12.000 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42078.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42078.log.gz 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-client.admin.56454.log 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38731.log 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56454.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56454.log.gz 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60693.log 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38731.log.gz 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53619.log 2024-05-08T03:23:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60693.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60693.log.gz 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-client.admin.58234.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53619.log.gz 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42638.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58234.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58234.log.gz 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-client.admin.60819.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42638.log.gz 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63045.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60819.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60819.log.gz 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-client.admin.38299.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63045.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60014.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63045.log.gz 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63213.log 2024-05-08T03:23:12.002 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38299.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.60014.log: -- /var/log/ceph/ceph-client.admin.60413.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60014.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63213.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56039.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60413.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60413.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr: -- /var/log/ceph/ceph-mon.c.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56039.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40758.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43576.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52890.log 2024-05-08T03:23:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43576.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59133.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43576.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52890.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54399.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38303.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59133.log: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.59133.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54399.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62919.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63129.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38303.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62919.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58954.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63129.log.gz 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61197.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45635.log 2024-05-08T03:23:12.004 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58954.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61197.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61197.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47835.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45635.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51481.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47835.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51481.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39115.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62793.log 2024-05-08T03:23:12.005 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39115.log.gz 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46573.log 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62793.log.gz 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53274.log 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46573.log.gz 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43076.log 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53274.log.gz 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62094.log 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43076.log.gz 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58176.log 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62094.log.gz 2024-05-08T03:23:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52071.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58176.log.gz 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62625.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52071.log.gz 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39157.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62625.log.gz 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53878.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39157.log.gz 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40478.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53878.log.gz 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56079.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.40478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40478.log.gz 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45854.log 2024-05-08T03:23:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56079.log.gz 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38021.log 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45854.log.gz 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46490.log 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38021.log.gz 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43133.log 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46490.log.gz 2024-05-08T03:23:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44093.log 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43133.log.gz 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60651.log 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44093.log.gz 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62457.log 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60651.log.gz 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62751.log 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62457.log.gz 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42896.log 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62751.log.gz 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38947.log 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42896.log.gz 2024-05-08T03:23:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38947.log.gz 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50215.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56497.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.50215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50215.log.gz 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52610.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56497.log.gz 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55608.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52610.log.gz 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51384.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55608.log.gz 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38763.log 2024-05-08T03:23:12.010 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51384.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38763.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62499.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62499.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61756.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62961.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61756.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49421.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62961.log.gz 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-05-08T03:23:12.011 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.49421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49421.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26334.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38354.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.tmp-client.admin.26334.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26334.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42815.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38354.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61463.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42815.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39740.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61463.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54158.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39740.log.gz 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55277.log 2024-05-08T03:23:12.012 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54158.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46266.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55277.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43853.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46266.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47795.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43853.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53334.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.47795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47795.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51766.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53334.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48732.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51766.log.gz 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-05-08T03:23:12.013 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48732.log.gz 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61455.log 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44182.log 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61455.log.gz 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53050.log 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44182.log.gz 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53050.log.gz 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61810.log 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61461.log 2024-05-08T03:23:12.014 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61810.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62667.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61461.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60414.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62667.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38017.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60414.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57331.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38017.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58475.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57331.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41075.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58475.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39073.log 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41075.log.gz 2024-05-08T03:23:12.015 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38014.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39073.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41580.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38014.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63297.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41580.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59849.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63297.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59849.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61071.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56836.log 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-08T03:23:12.016 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61071.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49420.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56836.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54275.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56836.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63171.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.49420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49420.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54275.log.gzgzip 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61113.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61281.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61113.log: /var/log/ceph/ceph-client.admin.63171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63171.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61113.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57096.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60735.log 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61281.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57096.log.gz 2024-05-08T03:23:12.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44673.log 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61758.log 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.44673.log: /var/log/ceph/ceph-client.admin.60735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60735.log.gz 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44673.log.gz 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55220.log 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63087.log 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61758.log.gz 2024-05-08T03:23:12.018 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55220.log.gz 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39199.log 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39261.log 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.63087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63087.log.gz 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39199.log.gz 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54295.log 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55658.log 2024-05-08T03:23:12.019 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39261.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54295.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58135.log 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54742.log 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55658.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58135.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61462.log 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37710.log 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61462.log: /var/log/ceph/ceph-client.admin.54742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54742.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61462.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52029.log 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61029.log 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37710.log.gz 2024-05-08T03:23:12.020 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52029.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51424.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51047.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.61029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.51424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51424.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54442.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60861.log: /var/log/ceph/ceph-client.admin.51047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59392.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.54442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54442.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59392.log.gz 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-05-08T03:23:12.021 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.56877.log: /var/log/ceph/ceph-client.admin.51966.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42516.log 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57533.log 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42516.log.gz 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38012.log 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59611.log 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38012.log: /var/log/ceph/ceph-client.admin.57533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57533.log.gz 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38012.log.gz 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60330.log 2024-05-08T03:23:12.022 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55064.log 2024-05-08T03:23:12.023 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.59611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59611.log.gz 2024-05-08T03:23:12.023 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60330.log.gz 2024-05-08T03:23:12.023 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55001.log 2024-05-08T03:23:12.023 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48628.log 2024-05-08T03:23:12.023 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.55001.log: /var/log/ceph/ceph-client.admin.55064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55064.log.gz 2024-05-08T03:23:12.023 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55001.log.gz 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60211.log 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50655.log 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.48628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48628.log.gz 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60211.log.gz 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62156.log 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58995.log 2024-05-08T03:23:12.024 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62156.log: /var/log/ceph/ceph-client.admin.50655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50655.log.gz 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62156.log.gz 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43472.log 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60777.log 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.58995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58995.log.gz 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.43472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43472.log.gz 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62541.log 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57015.log 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62541.log: /var/log/ceph/ceph-client.admin.60777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60777.log.gz 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62541.log.gz 2024-05-08T03:23:12.025 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49760.log 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.57015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57015.log.gz 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.41354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42235.log 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38611.log 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.42235.log: /var/log/ceph/ceph-client.admin.49760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49760.log.gz 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42235.log.gz 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52430.log 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61460.log 2024-05-08T03:23:12.026 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38611.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52430.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50037.log 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62709.log 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.50037.log: /var/log/ceph/ceph-client.admin.61460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61460.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50037.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45811.log 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.62709.log: /var/log/ceph/ceph-client.admin.53537.log 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62709.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.45811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62835.log 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.53537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45811.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53537.log.gz 2024-05-08T03:23:12.027 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32186.log 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62835.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62331.log 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62835.log.gz 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39398.log 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.32186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32186.log.gz 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62331.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62877.log 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62331.log.gz 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60609.log 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.39398.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39398.log.gz 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.62877.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-05-08T03:23:12.028 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62877.log.gz 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46368.log 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.60609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60609.log.gz 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38019.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38847.log 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.46368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60945.log 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46368.log.gz 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52849.log 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0%/var/log/ceph/ceph-client.admin.60945.log: -- replaced with /var/log/ceph/ceph-client.admin.38847.log.gz 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60945.log.gz 2024-05-08T03:23:12.029 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.52849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52849.log.gz 2024-05-08T03:23:12.030 INFO:teuthology.orchestra.run.smithi188.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T03:23:12.079 INFO:teuthology.orchestra.run.smithi177.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T03:23:12.285 INFO:teuthology.orchestra.run.smithi177.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T03:23:12.554 INFO:teuthology.orchestra.run.smithi188.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T03:23:12.698 INFO:teuthology.orchestra.run.smithi177.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T03:23:13.005 INFO:teuthology.orchestra.run.smithi188.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T03:23:13.140 INFO:teuthology.orchestra.run.smithi188.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T03:23:15.149 INFO:teuthology.orchestra.run.smithi177.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T03:23:15.862 INFO:teuthology.orchestra.run.smithi188.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T03:23:15.864 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-05-08T03:23:15.864 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m4.015s 2024-05-08T03:23:15.864 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m6.991s 2024-05-08T03:23:15.864 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.293s 2024-05-08T03:23:17.264 INFO:teuthology.orchestra.run.smithi177.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T03:23:18.311 INFO:teuthology.orchestra.run.smithi177.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T03:23:18.314 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-05-08T03:23:18.314 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m6.501s 2024-05-08T03:23:18.314 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m16.175s 2024-05-08T03:23:18.314 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m0.778s 2024-05-08T03:23:18.314 INFO:tasks.ceph:Archiving logs... 2024-05-08T03:23:18.314 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/remote/smithi177/log 2024-05-08T03:23:18.316 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T03:23:20.776 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/remote/smithi188/log 2024-05-08T03:23:20.777 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T03:23:21.745 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T03:23:21.755 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T03:23:21.755 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T03:23:21.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T03:23:21.845 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T03:23:21.855 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T03:23:21.855 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T03:23:21.858 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T03:23:21.872 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-05-08T03:23:21.875 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-08T03:23:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T03:23:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-05-08T03:23:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 44 +936us[ +936us] +/- 52ms 2024-05-08T03:23:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -944us[ -944us] +/- 60ms 2024-05-08T03:23:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -121us[ -120us] +/- 77ms 2024-05-08T03:23:21.877 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:23:21.881 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T03:23:21.881 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-08T03:23:21.881 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 45 +950us[ +950us] +/- 52ms 2024-05-08T03:23:21.881 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -928us[ -921us] +/- 60ms 2024-05-08T03:23:21.881 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -89us[ -81us] +/- 77ms 2024-05-08T03:23:21.881 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T03:23:21.881 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T03:23:21.890 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T03:23:21.891 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T03:23:21.900 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T03:23:22.003 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T03:23:22.080 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T03:23:22.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:23:22.111 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T03:23:22.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:23:22.141 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T03:23:22.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T03:23:22.167 INFO:teuthology.task.internal:Duration was 679.757741 seconds 2024-05-08T03:23:22.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T03:23:22.177 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T03:23:22.177 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T03:23:22.180 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T03:23:22.217 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T03:23:22.221 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T03:23:22.513 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T03:23:22.513 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-05-08T03:23:22.513 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T03:23:22.574 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-05-08T03:23:22.575 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T03:23:22.600 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T03:23:22.601 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T03:23:22.617 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T03:23:22.667 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T03:23:22.667 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T03:23:22.684 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T03:23:22.820 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T03:23:22.830 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T03:23:22.830 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T03:23:22.855 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T03:23:22.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T03:23:22.890 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T03:23:22.898 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T03:23:22.919 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-05-08T03:23:22.943 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-05-08T03:23:22.960 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T03:23:23.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:23:23.008 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T03:23:23.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:23:23.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T03:23:23.035 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T03:23:23.036 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/remote/smithi177 2024-05-08T03:23:23.036 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T03:23:23.089 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-05-07_23:02:08-rbd-wip-dis-testing-squid-distro-default-smithi/7696770/remote/smithi188 2024-05-08T03:23:23.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T03:23:23.123 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T03:23:23.123 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T03:23:23.126 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T03:23:23.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T03:23:23.184 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T03:23:23.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T03:23:23.194 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T03:23:23.194 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T03:23:23.196 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T03:23:23.210 INFO:teuthology.orchestra.run.smithi177.stdout: 265822 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 03:23 /home/ubuntu/cephtest 2024-05-08T03:23:23.227 INFO:teuthology.orchestra.run.smithi188.stdout: 265518 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 03:23 /home/ubuntu/cephtest 2024-05-08T03:23:23.228 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T03:23:23.239 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T03:23:23.262 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 679.7577412128448 flavor: default owner: scheduled_dis@teuthology success: true 2024-05-08T03:23:23.262 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T03:23:23.345 INFO:teuthology.run:pass